2024-08-18T13:26:49.179 INFO:root:teuthology version: 1.2.0 2024-08-18T13:26:49.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-18T13:26:49.293 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093 branch: wip-mchangir-mgr-volumes-print-thread-name-quincy description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: mchangir@redhat.com first_in_suite: false job_id: '7859093' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - filesystem is online with fewer MDS than max_mds - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum - Replacing daemon mds sha1: a17c67c052c1476b20f99d847823755b26703128 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a17c67c052c1476b20f99d847823755b26703128 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy sha1: a17c67c052c1476b20f99d847823755b26703128 owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 5038 sha1: a17c67c052c1476b20f99d847823755b26703128 sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-mgr-volumes-print-thread-name-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a17c67c052c1476b20f99d847823755b26703128 targets: smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - mds_thrash: null - workunit: clients: all: - fs/snaps teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-18_13:11:53 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-18T13:26:49.293 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa; will attempt to use it 2024-08-18T13:26:49.294 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa/tasks 2024-08-18T13:26:49.294 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-18T13:26:49.296 INFO:teuthology.task.internal:Checking packages... 2024-08-18T13:26:49.316 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'a17c67c052c1476b20f99d847823755b26703128' 2024-08-18T13:26:49.317 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-18T13:26:49.317 INFO:teuthology.packaging:ref: None 2024-08-18T13:26:49.317 INFO:teuthology.packaging:tag: None 2024-08-18T13:26:49.317 INFO:teuthology.packaging:branch: wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-18T13:26:49.317 INFO:teuthology.packaging:sha1: a17c67c052c1476b20f99d847823755b26703128 2024-08-18T13:26:49.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-18T13:26:49.490 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:26:49.492 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-18T13:26:49.498 INFO:teuthology.task.internal:no buildpackages task found 2024-08-18T13:26:49.499 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-18T13:26:49.514 INFO:teuthology.task.internal:Saving configuration 2024-08-18T13:26:49.548 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-18T13:26:49.556 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-18T13:26:49.578 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-18 13:20:09.039196', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-18T13:26:49.600 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-18 13:20:09.037368', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-18T13:26:49.601 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-18T13:26:49.607 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-08-18T13:26:49.607 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-08-18T13:26:49.607 INFO:teuthology.run_tasks:Running task console_log... 2024-08-18T13:26:49.661 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7640433ac0>, signals=[15]) 2024-08-18T13:26:49.661 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-18T13:26:49.667 INFO:teuthology.task.internal:Opening connections... 2024-08-18T13:26:49.667 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:26:49.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:26:49.766 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:26:49.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:26:49.862 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-18T13:26:49.869 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-08-18T13:26:49.876 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-08-18T13:26:49.876 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-08-18T13:26:49.925 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Ubuntu" 2024-08-18T13:26:49.925 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-18T13:26:49.925 INFO:teuthology.orchestra.run.smithi118.stdout:ID=ubuntu 2024-08-18T13:26:49.925 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE=debian 2024-08-18T13:26:49.925 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-18T13:26:49.925 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="20.04" 2024-08-18T13:26:49.925 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-18T13:26:49.926 INFO:teuthology.orchestra.run.smithi118.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-18T13:26:49.926 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-18T13:26:49.926 INFO:teuthology.orchestra.run.smithi118.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-18T13:26:49.926 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_CODENAME=focal 2024-08-18T13:26:49.926 INFO:teuthology.orchestra.run.smithi118.stdout:UBUNTU_CODENAME=focal 2024-08-18T13:26:49.926 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-08-18T13:26:49.946 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-08-18T13:26:49.953 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-08-18T13:26:49.953 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-08-18T13:26:50.002 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="20.04" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=focal 2024-08-18T13:26:50.003 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=focal 2024-08-18T13:26:50.004 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-08-18T13:26:50.022 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-18T13:26:50.031 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-18T13:26:50.037 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-18T13:26:50.037 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2024-08-18T13:26:50.039 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-08-18T13:26:50.045 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-18T13:26:50.051 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-18T13:26:50.051 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2024-08-18T13:26:50.054 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-08-18T13:26:50.112 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-18T13:26:50.167 INFO:teuthology.run_tasks:Running task kernel... 2024-08-18T13:26:50.181 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-18T13:26:50.181 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-18T13:26:50.181 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-18T13:26:50.182 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-18T13:26:50.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:26:50.188 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-08-18T13:26:50.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:26:50.189 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-08-18T13:26:50.195 INFO:teuthology.orchestra.run.smithi118.stdout:5.4.0-124-generic 2024-08-18T13:26:50.195 INFO:teuthology.task.kernel:Running kernel on smithi118: 5.4.0-124-generic 2024-08-18T13:26:50.195 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get clean 2024-08-18T13:26:50.196 INFO:teuthology.orchestra.run.smithi178.stdout:5.4.0-124-generic 2024-08-18T13:26:50.196 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.4.0-124-generic 2024-08-18T13:26:50.196 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get clean 2024-08-18T13:26:50.357 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-08-18T13:26:50.477 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-08-18T13:26:50.524 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-18T13:26:50.634 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-18T13:26:50.642 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-18T13:26:50.716 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-18T13:26:50.760 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-18T13:26:50.814 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,126 kB] 2024-08-18T13:26:50.841 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-18T13:26:50.937 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [797 kB] 2024-08-18T13:26:50.978 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [797 kB] 2024-08-18T13:26:50.994 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-18T13:26:50.998 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.2 kB] 2024-08-18T13:26:50.998 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-18T13:26:50.998 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-18T13:26:51.020 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,126 kB] 2024-08-18T13:26:51.023 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-18T13:26:51.026 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-18T13:26:51.026 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-18T13:26:51.030 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-18T13:26:51.037 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-18T13:26:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.0 kB] 2024-08-18T13:26:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-18T13:26:51.039 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-18T13:26:51.039 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-18T13:26:51.045 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-18T13:26:51.064 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-18T13:26:51.069 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-18T13:26:51.069 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.2 kB] 2024-08-18T13:26:51.070 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-18T13:26:51.070 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-18T13:26:51.099 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-18T13:26:51.102 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-18T13:26:51.102 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-18T13:26:51.106 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-18T13:26:51.114 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-18T13:26:51.115 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.0 kB] 2024-08-18T13:26:51.115 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-18T13:26:51.115 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-18T13:26:51.115 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-18T13:26:51.123 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-18T13:26:51.193 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-18T13:26:51.210 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,017 kB] 2024-08-18T13:26:51.333 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,017 kB] 2024-08-18T13:26:51.453 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,496 kB] 2024-08-18T13:26:51.577 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,496 kB] 2024-08-18T13:26:51.630 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-18T13:26:51.650 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-18T13:26:51.650 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-18T13:26:51.651 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-18T13:26:51.744 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-18T13:26:51.746 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-18T13:26:51.752 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-18T13:26:51.752 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-18T13:26:51.759 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-18T13:26:51.760 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-18T13:26:51.761 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-18T13:26:51.787 INFO:teuthology.orchestra.run.smithi178.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-18T13:26:51.813 INFO:teuthology.orchestra.run.smithi178.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-18T13:26:51.819 INFO:teuthology.orchestra.run.smithi178.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-18T13:26:51.820 INFO:teuthology.orchestra.run.smithi178.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-18T13:26:51.821 INFO:teuthology.orchestra.run.smithi178.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-18T13:26:51.821 INFO:teuthology.orchestra.run.smithi178.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-18T13:26:51.849 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-18T13:26:51.866 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-18T13:26:51.866 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-18T13:26:51.882 INFO:teuthology.orchestra.run.smithi178.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-18T13:26:51.882 INFO:teuthology.orchestra.run.smithi178.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-18T13:26:51.883 INFO:teuthology.orchestra.run.smithi178.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-18T13:26:51.884 INFO:teuthology.orchestra.run.smithi178.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-18T13:26:51.894 INFO:teuthology.orchestra.run.smithi118.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-18T13:26:51.922 INFO:teuthology.orchestra.run.smithi118.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-18T13:26:51.930 INFO:teuthology.orchestra.run.smithi118.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-18T13:26:51.933 INFO:teuthology.orchestra.run.smithi118.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-18T13:26:51.934 INFO:teuthology.orchestra.run.smithi118.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-18T13:26:51.934 INFO:teuthology.orchestra.run.smithi118.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-18T13:26:51.963 INFO:teuthology.orchestra.run.smithi178.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-18T13:26:51.963 INFO:teuthology.orchestra.run.smithi178.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-18T13:26:51.964 INFO:teuthology.orchestra.run.smithi178.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-18T13:26:51.991 INFO:teuthology.orchestra.run.smithi118.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-18T13:26:51.991 INFO:teuthology.orchestra.run.smithi118.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-18T13:26:51.992 INFO:teuthology.orchestra.run.smithi118.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-18T13:26:51.993 INFO:teuthology.orchestra.run.smithi118.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-18T13:26:52.072 INFO:teuthology.orchestra.run.smithi118.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-18T13:26:52.073 INFO:teuthology.orchestra.run.smithi118.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-18T13:26:52.074 INFO:teuthology.orchestra.run.smithi118.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-18T13:26:55.329 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 21.4 MB in 3s (7,151 kB/s) 2024-08-18T13:26:55.481 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 21.4 MB in 3s (7,438 kB/s) 2024-08-18T13:26:56.852 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-08-18T13:26:56.878 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-18T13:26:56.950 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-08-18T13:26:57.074 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-08-18T13:26:57.100 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-18T13:26:57.165 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-08-18T13:26:57.165 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-08-18T13:26:57.172 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-08-18T13:26:57.376 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-08-18T13:26:57.376 INFO:teuthology.orchestra.run.smithi178.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-18T13:26:57.376 INFO:teuthology.orchestra.run.smithi178.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-18T13:26:57.376 INFO:teuthology.orchestra.run.smithi178.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-18T13:26:57.376 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-08-18T13:26:57.376 INFO:teuthology.orchestra.run.smithi178.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-18T13:26:57.376 INFO:teuthology.orchestra.run.smithi178.stdout:Recommended packages: 2024-08-18T13:26:57.377 INFO:teuthology.orchestra.run.smithi178.stdout: thermald 2024-08-18T13:26:57.384 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-08-18T13:26:57.385 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-08-18T13:26:57.437 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-08-18T13:26:57.437 INFO:teuthology.orchestra.run.smithi178.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-18T13:26:57.437 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-18T13:26:57.437 INFO:teuthology.orchestra.run.smithi178.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-18T13:26:57.438 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages will be upgraded: 2024-08-18T13:26:57.438 INFO:teuthology.orchestra.run.smithi178.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-18T13:26:57.473 INFO:teuthology.orchestra.run.smithi178.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-08-18T13:26:57.473 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 77.2 MB of archives. 2024-08-18T13:26:57.473 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-18T13:26:57.473 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-18T13:26:57.578 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2024-08-18T13:26:57.578 INFO:teuthology.orchestra.run.smithi118.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-18T13:26:57.578 INFO:teuthology.orchestra.run.smithi118.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-18T13:26:57.578 INFO:teuthology.orchestra.run.smithi118.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-18T13:26:57.579 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2024-08-18T13:26:57.579 INFO:teuthology.orchestra.run.smithi118.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-18T13:26:57.579 INFO:teuthology.orchestra.run.smithi118.stdout:Recommended packages: 2024-08-18T13:26:57.580 INFO:teuthology.orchestra.run.smithi118.stdout: thermald 2024-08-18T13:26:57.647 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2024-08-18T13:26:57.647 INFO:teuthology.orchestra.run.smithi118.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-18T13:26:57.647 INFO:teuthology.orchestra.run.smithi118.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-18T13:26:57.647 INFO:teuthology.orchestra.run.smithi118.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-18T13:26:57.649 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages will be upgraded: 2024-08-18T13:26:57.649 INFO:teuthology.orchestra.run.smithi118.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-18T13:26:57.810 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-18T13:26:57.832 INFO:teuthology.orchestra.run.smithi118.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-08-18T13:26:57.833 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 77.2 MB of archives. 2024-08-18T13:26:57.833 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-18T13:26:57.833 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-18T13:26:57.903 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-18T13:26:58.385 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-18T13:26:58.385 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-18T13:26:58.385 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-18T13:26:58.479 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-18T13:26:58.489 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-18T13:26:58.985 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-18T13:26:59.181 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 77.2 MB in 1s (72.7 MB/s) 2024-08-18T13:26:59.257 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-18T13:26:59.317 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-18T13:27:00.697 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-18T13:27:00.697 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-18T13:27:00.698 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-18T13:27:00.972 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-18T13:27:01.036 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-18T13:27:01.067 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-18T13:27:01.072 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:01.148 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:01.844 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 77.2 MB in 3s (22.8 MB/s) 2024-08-18T13:27:02.023 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-18T13:27:03.149 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-18T13:27:03.164 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:03.258 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:03.770 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-18T13:27:03.775 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:03.979 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:04.336 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-18T13:27:04.351 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:04.394 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:05.897 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-18T13:27:05.914 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:06.006 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:07.142 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-18T13:27:07.155 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:07.184 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:09.241 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-18T13:27:09.402 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-18T13:27:09.745 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-18T13:27:09.872 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-18T13:27:10.225 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-18T13:27:10.238 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-18T13:27:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-18T13:27:12.334 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-18T13:27:12.461 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-18T13:27:12.802 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-18T13:27:12.923 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-18T13:27:13.268 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-18T13:27:13.284 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-18T13:27:13.326 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-18T13:27:15.551 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-18T13:27:15.574 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:15.617 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:18.425 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-18T13:27:18.548 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-18T13:27:18.889 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-18T13:27:19.002 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:19.485 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-18T13:27:19.508 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-18T13:27:19.551 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:20.239 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:21.396 INFO:teuthology.orchestra.run.smithi178.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-18T13:27:21.397 INFO:teuthology.orchestra.run.smithi178.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:21.541 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:21.667 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:22.246 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-18T13:27:22.359 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-18T13:27:22.718 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-18T13:27:22.835 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-18T13:27:22.838 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:22.961 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-18T13:27:23.088 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-18T13:27:23.214 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:23.280 INFO:teuthology.orchestra.run.smithi178.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-18T13:27:23.280 INFO:teuthology.orchestra.run.smithi178.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:24.407 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:25.560 INFO:teuthology.orchestra.run.smithi118.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-18T13:27:25.561 INFO:teuthology.orchestra.run.smithi118.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:25.726 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:25.852 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:27.078 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-18T13:27:27.204 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-18T13:27:27.330 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-18T13:27:27.490 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-18T13:27:27.576 INFO:teuthology.orchestra.run.smithi118.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-18T13:27:27.576 INFO:teuthology.orchestra.run.smithi118.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:45.983 INFO:teuthology.orchestra.run.smithi178.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-18T13:27:45.983 INFO:teuthology.orchestra.run.smithi178.stdout:Sourcing file `/etc/default/grub' 2024-08-18T13:27:45.997 INFO:teuthology.orchestra.run.smithi178.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-18T13:27:46.005 INFO:teuthology.orchestra.run.smithi178.stdout:Generating grub configuration file ... 2024-08-18T13:27:46.353 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-18T13:27:46.383 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:46.754 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-18T13:27:46.764 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-18T13:27:48.566 INFO:teuthology.orchestra.run.smithi178.stdout:done 2024-08-18T13:27:48.996 DEBUG:teuthology.orchestra.run.smithi178:> dpkg -s linux-image-generic 2024-08-18T13:27:49.043 INFO:teuthology.orchestra.run.smithi178.stdout:Package: linux-image-generic 2024-08-18T13:27:49.043 INFO:teuthology.orchestra.run.smithi178.stdout:Status: install ok installed 2024-08-18T13:27:49.043 INFO:teuthology.orchestra.run.smithi178.stdout:Priority: optional 2024-08-18T13:27:49.043 INFO:teuthology.orchestra.run.smithi178.stdout:Section: kernel 2024-08-18T13:27:49.043 INFO:teuthology.orchestra.run.smithi178.stdout:Installed-Size: 20 2024-08-18T13:27:49.043 INFO:teuthology.orchestra.run.smithi178.stdout:Maintainer: Ubuntu Kernel Team 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: amd64 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout:Source: linux-meta 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout:Version: 5.4.0.192.190 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout:Recommends: thermald 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout:Description: Generic Linux kernel image 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout: This package will always depend on the latest generic kernel image 2024-08-18T13:27:49.044 INFO:teuthology.orchestra.run.smithi178.stdout: available. 2024-08-18T13:27:49.044 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-18T13:27:49.045 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-18T13:27:49.045 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-18T13:27:49.045 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-08-18T13:27:49.045 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-08-18T13:27:49.045 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get clean 2024-08-18T13:27:49.117 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-08-18T13:27:49.260 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-18T13:27:49.260 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-18T13:27:49.265 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-18T13:27:49.283 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-18T13:27:50.032 INFO:teuthology.orchestra.run.smithi118.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-18T13:27:50.032 INFO:teuthology.orchestra.run.smithi118.stdout:Sourcing file `/etc/default/grub' 2024-08-18T13:27:50.039 INFO:teuthology.orchestra.run.smithi118.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-18T13:27:50.050 INFO:teuthology.orchestra.run.smithi118.stdout:Generating grub configuration file ... 2024-08-18T13:27:50.351 INFO:teuthology.orchestra.run.smithi118.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-18T13:27:50.376 INFO:teuthology.orchestra.run.smithi118.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:50.794 INFO:teuthology.orchestra.run.smithi118.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-18T13:27:50.806 INFO:teuthology.orchestra.run.smithi118.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-18T13:27:51.104 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-08-18T13:27:51.129 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-18T13:27:51.204 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-08-18T13:27:51.414 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-08-18T13:27:51.415 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-08-18T13:27:51.643 INFO:teuthology.orchestra.run.smithi178.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-18T13:27:51.644 INFO:teuthology.orchestra.run.smithi178.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-08-18T13:27:51.645 DEBUG:teuthology.orchestra.run.smithi178:> dpkg -s linux-image-generic 2024-08-18T13:27:51.667 INFO:teuthology.orchestra.run.smithi178.stdout:Package: linux-image-generic 2024-08-18T13:27:51.667 INFO:teuthology.orchestra.run.smithi178.stdout:Status: install ok installed 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Priority: optional 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Section: kernel 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Installed-Size: 20 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Maintainer: Ubuntu Kernel Team 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: amd64 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Source: linux-meta 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Version: 5.4.0.192.190 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Recommends: thermald 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout:Description: Generic Linux kernel image 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: This package will always depend on the latest generic kernel image 2024-08-18T13:27:51.668 INFO:teuthology.orchestra.run.smithi178.stdout: available. 2024-08-18T13:27:51.669 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-08-18T13:27:51.675 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.EzIwaQ5xd9 2024-08-18T13:27:51.675 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub/grub.cfg /tmp/tmp.EzIwaQ5xd9 2024-08-18T13:27:51.732 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.EzIwaQ5xd9 2024-08-18T13:27:51.767 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.EzIwaQ5xd9 is 9KB 2024-08-18T13:27:51.782 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.EzIwaQ5xd9 2024-08-18T13:27:51.788 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-18T13:27:51.843 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:27:51.843 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-18T13:27:51.843 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-18T13:27:51.875 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-18T13:27:51.875 DEBUG:teuthology.orchestra.run.smithi178:> sudo update-grub 2024-08-18T13:27:53.017 INFO:teuthology.orchestra.run.smithi118.stdout:done 2024-08-18T13:27:53.218 INFO:teuthology.orchestra.run.smithi178.stderr:Sourcing file `/etc/default/grub' 2024-08-18T13:27:53.223 INFO:teuthology.orchestra.run.smithi178.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-18T13:27:53.243 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-08-18T13:27:53.464 DEBUG:teuthology.orchestra.run.smithi118:> dpkg -s linux-image-generic 2024-08-18T13:27:53.505 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-18T13:27:53.517 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:53.518 INFO:teuthology.orchestra.run.smithi118.stdout:Package: linux-image-generic 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Status: install ok installed 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Priority: optional 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Section: kernel 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Installed-Size: 20 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Maintainer: Ubuntu Kernel Team 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: amd64 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Source: linux-meta 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Version: 5.4.0.192.190 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Recommends: thermald 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout:Description: Generic Linux kernel image 2024-08-18T13:27:53.519 INFO:teuthology.orchestra.run.smithi118.stdout: This package will always depend on the latest generic kernel image 2024-08-18T13:27:53.520 INFO:teuthology.orchestra.run.smithi118.stdout: available. 2024-08-18T13:27:53.520 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-18T13:27:53.520 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-18T13:27:53.520 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-18T13:27:53.520 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-08-18T13:27:53.520 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi118.front.sepia.ceph.com, path=None, version=distro) 2024-08-18T13:27:53.520 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get clean 2024-08-18T13:27:53.602 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-08-18T13:27:53.741 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-18T13:27:53.747 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-18T13:27:53.764 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-18T13:27:53.782 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-18T13:27:53.894 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-18T13:27:53.905 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-18T13:27:54.652 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-08-18T13:27:54.653 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-08-18T13:27:55.603 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-08-18T13:27:55.630 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-18T13:27:55.700 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-08-18T13:27:55.889 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-08-18T13:27:55.889 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-08-18T13:27:56.114 INFO:teuthology.orchestra.run.smithi118.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-18T13:27:56.114 INFO:teuthology.orchestra.run.smithi118.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-08-18T13:27:56.116 DEBUG:teuthology.orchestra.run.smithi118:> dpkg -s linux-image-generic 2024-08-18T13:27:56.138 INFO:teuthology.orchestra.run.smithi118.stdout:Package: linux-image-generic 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Status: install ok installed 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Priority: optional 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Section: kernel 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Installed-Size: 20 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Maintainer: Ubuntu Kernel Team 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: amd64 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Source: linux-meta 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Version: 5.4.0.192.190 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Recommends: thermald 2024-08-18T13:27:56.139 INFO:teuthology.orchestra.run.smithi118.stdout:Description: Generic Linux kernel image 2024-08-18T13:27:56.140 INFO:teuthology.orchestra.run.smithi118.stdout: This package will always depend on the latest generic kernel image 2024-08-18T13:27:56.140 INFO:teuthology.orchestra.run.smithi118.stdout: available. 2024-08-18T13:27:56.140 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-08-18T13:27:56.146 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.0vxR72FrCD 2024-08-18T13:27:56.146 DEBUG:teuthology.orchestra.run.smithi118:> sudo cp /boot/grub/grub.cfg /tmp/tmp.0vxR72FrCD 2024-08-18T13:27:56.203 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.0vxR72FrCD 2024-08-18T13:27:56.240 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.0vxR72FrCD is 9KB 2024-08-18T13:27:56.257 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.0vxR72FrCD 2024-08-18T13:27:56.263 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-18T13:27:56.322 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:27:56.322 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-18T13:27:56.322 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-18T13:27:56.367 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-18T13:27:56.367 DEBUG:teuthology.orchestra.run.smithi118:> sudo update-grub 2024-08-18T13:27:57.680 INFO:teuthology.orchestra.run.smithi118.stderr:Sourcing file `/etc/default/grub' 2024-08-18T13:27:57.703 INFO:teuthology.orchestra.run.smithi118.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-18T13:27:57.715 INFO:teuthology.orchestra.run.smithi118.stderr:Generating grub configuration file ... 2024-08-18T13:27:57.965 INFO:teuthology.orchestra.run.smithi118.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-18T13:27:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-18T13:27:58.380 INFO:teuthology.orchestra.run.smithi118.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-18T13:27:58.392 INFO:teuthology.orchestra.run.smithi118.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-18T13:27:59.302 INFO:teuthology.orchestra.run.smithi118.stderr:done 2024-08-18T13:27:59.304 DEBUG:teuthology.orchestra.run.smithi118:> sudo shutdown -r now 2024-08-18T13:28:24.681 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-08-18T13:28:24.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-08-18T13:28:24.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:28:29.310 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-08-18T13:28:29.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-08-18T13:28:29.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:28:59.363 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-08-18T13:29:03.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-08-18T13:29:08.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-08-18T13:29:08.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:29:12.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-08-18T13:29:12.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:29:16.031 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-08-18T13:29:28.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-08-18T13:29:28.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:29:59.797 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-08-18T13:30:00.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-08-18T13:30:00.412 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-192-generic"... 2024-08-18T13:30:00.412 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-08-18T13:30:00.418 INFO:teuthology.orchestra.run.smithi118.stdout:5.4.0-192-generic 2024-08-18T13:30:00.418 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-18T13:30:00.419 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-18T13:30:00.419 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-18T13:30:01.419 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-18T13:30:01.420 DEBUG:teuthology.orchestra.run.smithi118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-18T13:30:01.510 INFO:teuthology.orchestra.run.smithi118.stdout:ttyS1 2024-08-18T13:30:01.535 DEBUG:teuthology.parallel:result is None 2024-08-18T13:30:08.372 DEBUG:teuthology.orchestra.remote:timed out 2024-08-18T13:30:20.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-08-18T13:30:20.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:30:20.614 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-08-18T13:30:21.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-08-18T13:30:21.276 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-192-generic"... 2024-08-18T13:30:21.276 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-08-18T13:30:21.282 INFO:teuthology.orchestra.run.smithi178.stdout:5.4.0-192-generic 2024-08-18T13:30:21.282 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-18T13:30:21.282 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-18T13:30:21.283 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-18T13:30:22.284 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-18T13:30:22.284 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-18T13:30:22.373 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-08-18T13:30:22.400 DEBUG:teuthology.parallel:result is None 2024-08-18T13:30:22.401 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-18T13:30:22.418 INFO:teuthology.task.internal:Creating test directory... 2024-08-18T13:30:22.418 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-18T13:30:22.420 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-18T13:30:22.427 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-18T13:30:22.434 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-18T13:30:22.443 INFO:teuthology.task.internal:Creating archive directory... 2024-08-18T13:30:22.443 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-18T13:30:22.446 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-18T13:30:22.481 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-18T13:30:22.489 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-18T13:30:22.489 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-18T13:30:22.491 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-18T13:30:22.509 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T13:30:22.515 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T13:30:22.521 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T13:30:22.526 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T13:30:22.528 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-18T13:30:22.535 INFO:teuthology.task.internal:Configuring sudo... 2024-08-18T13:30:22.536 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-18T13:30:22.574 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-18T13:30:22.589 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-18T13:30:22.599 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-18T13:30:22.600 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-18T13:30:22.602 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-18T13:30:22.609 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T13:30:22.616 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T13:30:22.667 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:30:22.667 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-18T13:30:22.692 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T13:30:22.699 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T13:30:22.749 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:30:22.749 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-18T13:30:22.771 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2024-08-18T13:30:22.774 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-08-18T13:30:22.826 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-18T13:30:22.833 INFO:teuthology.task.internal:Starting timer... 2024-08-18T13:30:22.833 INFO:teuthology.run_tasks:Running task pcp... 2024-08-18T13:30:22.848 INFO:teuthology.run_tasks:Running task selinux... 2024-08-18T13:30:22.859 DEBUG:teuthology.task.selinux:Excluding smithi118: OS 'ubuntu' does not support SELinux 2024-08-18T13:30:22.859 DEBUG:teuthology.task.selinux:Excluding smithi178: OS 'ubuntu' does not support SELinux 2024-08-18T13:30:22.859 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-18T13:30:22.859 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-08-18T13:30:22.859 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-18T13:30:22.859 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-18T13:30:22.868 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-18T13:30:22.879 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-18T13:30:22.880 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi118.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-18T13:35:57.206 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi118.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-08-18T13:35:57.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-08-18T13:35:57.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:35:57.312 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-08-18T13:35:57.431 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-08-18T13:35:57.432 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-08-18T13:35:57.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T13:35:57.520 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-08-18T13:35:57.644 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-08-18T13:35:57.644 INFO:teuthology.run_tasks:Running task clock... 2024-08-18T13:35:57.659 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-18T13:35:57.659 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-18T13:35:57.660 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-18T13:35:57.663 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-18T13:35:57.663 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-18T13:35:57.695 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-18T13:35:57.696 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Command line: ntpd -gq 2024-08-18T13:35:57.697 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: proto: precision = 0.116 usec (-23) 2024-08-18T13:35:57.698 INFO:teuthology.orchestra.run.smithi118.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-18T13:35:57.698 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-18T13:35:57.698 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Command line: ntpd -gq 2024-08-18T13:35:57.699 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: proto: precision = 0.042 usec (-24) 2024-08-18T13:35:57.699 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-18T13:35:57.700 INFO:teuthology.orchestra.run.smithi178.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-18T13:35:57.700 INFO:teuthology.orchestra.run.smithi118.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-18T13:35:57.701 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-18T13:35:57.701 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: restrict ::: KOD does nothing without LIMITED. 2024-08-18T13:35:57.701 INFO:teuthology.orchestra.run.smithi178.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-18T13:35:57.703 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: restrict ::: KOD does nothing without LIMITED. 2024-08-18T13:35:57.703 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Listen and drop on 0 v6wildcard [::]:123 2024-08-18T13:35:57.703 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-18T13:35:57.703 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Listen normally on 2 lo 127.0.0.1:123 2024-08-18T13:35:57.704 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Listen normally on 3 enp3s0f1 172.21.15.118:123 2024-08-18T13:35:57.704 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Listen normally on 4 lo [::1]:123 2024-08-18T13:35:57.704 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d725%5]:123 2024-08-18T13:35:57.704 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:35:57 ntpd[13681]: Listening on routing socket on fd #22 for interface updates 2024-08-18T13:35:57.705 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Listen and drop on 0 v6wildcard [::]:123 2024-08-18T13:35:57.705 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-18T13:35:57.705 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Listen normally on 2 lo 127.0.0.1:123 2024-08-18T13:35:57.705 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Listen normally on 3 enp3s0f1 172.21.15.178:123 2024-08-18T13:35:57.705 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Listen normally on 4 lo [::1]:123 2024-08-18T13:35:57.705 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ef9%5]:123 2024-08-18T13:35:57.705 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:35:57 ntpd[13640]: Listening on routing socket on fd #22 for interface updates 2024-08-18T13:35:58.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:58.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:58.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:58.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:58.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:58.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:58.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:35:58.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:58 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:35:59.696 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:59 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:59.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:59 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:59.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:59 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:59.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:59 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:59.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:59 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:35:59.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:35:59 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:35:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:35:59.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:35:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:35:59.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:35:59 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:00.696 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:00 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:00.696 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:00 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:00.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:00 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:00.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:00 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:00.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:00 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:00.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:00 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:00.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:00 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:00.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:00 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:00.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:00 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:00.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:00 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:00.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:00 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:00.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:00 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:01.696 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:01 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:01.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:01 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:01.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:01 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:01.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:01 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:01.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:01 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:01.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:01 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:01.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:01.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:01.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:01 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:02.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:02.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:02.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:02.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:02.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:02.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:02.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:02.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:02.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:02.699 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:02.699 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:02.699 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:02 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:02.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:02 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:02.701 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:02 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:02.701 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:02 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:02.701 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:02 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:02.701 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:02 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:02.701 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:02 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:03.696 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:03 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:03.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:03 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:03.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:03 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:03.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:03 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:03.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:03 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:03.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:03 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:03.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:03.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:03.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:03.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:03.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:03 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:04.697 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:04.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:04.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:04.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:04.698 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:04.698 INFO:teuthology.orchestra.run.smithi118.stdout:18 Aug 13:36:04 ntpd[13681]: ntpd: time slew -0.000673 s 2024-08-18T13:36:04.698 INFO:teuthology.orchestra.run.smithi118.stdout:ntpd: time slew -0.000673s 2024-08-18T13:36:04.699 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-18T13:36:04.699 INFO:teuthology.orchestra.run.smithi118.stderr:18 Aug 13:36:04 ntpd[13681]: can't open /var/log/ntpstats/loopstats.20240818: Permission denied 2024-08-18T13:36:04.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:04 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:04.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:04 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:04.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:04 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:04.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:04 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:04.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:04 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:04.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:04 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:04.768 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2024-08-18T13:36:04.769 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2024-08-18T13:36:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:04.769 INFO:teuthology.orchestra.run.smithi118.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:05.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:05.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:05.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:05.698 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:05.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-18T13:36:05.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: can't open /var/log/ntpstats/rawstats.20240818: Permission denied 2024-08-18T13:36:05.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-18T13:36:05.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: can't open /var/log/ntpstats/peerstats.20240818: Permission denied 2024-08-18T13:36:05.699 INFO:teuthology.orchestra.run.smithi178.stdout:18 Aug 13:36:05 ntpd[13640]: ntpd: time slew +0.000614 s 2024-08-18T13:36:05.699 INFO:teuthology.orchestra.run.smithi178.stdout:ntpd: time slew +0.000614s 2024-08-18T13:36:05.699 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-18T13:36:05.700 INFO:teuthology.orchestra.run.smithi178.stderr:18 Aug 13:36:05 ntpd[13640]: can't open /var/log/ntpstats/loopstats.20240818: Permission denied 2024-08-18T13:36:05.769 INFO:teuthology.orchestra.run.smithi178.stdout: remote refid st t when poll reach delay offset jitter 2024-08-18T13:36:05.769 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================== 2024-08-18T13:36:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-18T13:36:05.771 INFO:teuthology.run_tasks:Running task install... 2024-08-18T13:36:05.782 DEBUG:teuthology.task.install:project ceph 2024-08-18T13:36:05.782 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a17c67c052c1476b20f99d847823755b26703128'}} 2024-08-18T13:36:05.782 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'a17c67c052c1476b20f99d847823755b26703128'} 2024-08-18T13:36:05.782 INFO:teuthology.task.install:Using flavor: default 2024-08-18T13:36:05.788 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-18T13:36:05.788 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-08-18T13:36:05.788 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-key list | grep Ceph 2024-08-18T13:36:05.789 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-key list | grep Ceph 2024-08-18T13:36:05.888 INFO:teuthology.orchestra.run.smithi178.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-18T13:36:05.890 INFO:teuthology.orchestra.run.smithi118.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-18T13:36:05.948 INFO:teuthology.orchestra.run.smithi178.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-18T13:36:05.949 INFO:teuthology.orchestra.run.smithi178.stdout:uid [ unknown] Ceph.com (release key) 2024-08-18T13:36:05.950 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-18T13:36:05.950 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-18T13:36:05.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a17c67c052c1476b20f99d847823755b26703128 2024-08-18T13:36:05.955 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-18T13:36:05.956 INFO:teuthology.orchestra.run.smithi118.stdout:uid [ unknown] Ceph.com (release key) 2024-08-18T13:36:05.956 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-18T13:36:05.956 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-18T13:36:05.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a17c67c052c1476b20f99d847823755b26703128 2024-08-18T13:36:06.091 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default/ 2024-08-18T13:36:06.091 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:36:06.106 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default/ 2024-08-18T13:36:06.107 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:36:06.318 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:36:06.318 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-18T13:36:06.333 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:36:06.333 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-18T13:36:06.335 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-08-18T13:36:06.350 DEBUG:teuthology.orchestra.run.smithi118:> sudo apt-get update 2024-08-18T13:36:06.494 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-18T13:36:06.499 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-18T13:36:06.517 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-18T13:36:06.629 INFO:teuthology.orchestra.run.smithi118.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-18T13:36:06.629 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal InRelease 2024-08-18T13:36:06.652 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-18T13:36:06.652 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-18T13:36:06.680 INFO:teuthology.orchestra.run.smithi178.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal InRelease 2024-08-18T13:36:06.683 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-18T13:36:06.723 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-18T13:36:06.734 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-18T13:36:06.740 INFO:teuthology.orchestra.run.smithi118.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal Release.gpg 2024-08-18T13:36:06.791 INFO:teuthology.orchestra.run.smithi178.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal Release.gpg 2024-08-18T13:36:06.795 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-18T13:36:06.805 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-18T13:36:06.845 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-18T13:36:06.852 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main i386 Packages [3,367 B] 2024-08-18T13:36:06.908 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main i386 Packages [3,367 B] 2024-08-18T13:36:07.151 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 28.4 kB in 1s (44.3 kB/s) 2024-08-18T13:36:07.298 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 28.4 kB in 1s (37.4 kB/s) 2024-08-18T13:36:08.293 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-08-18T13:36:08.318 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-ga17c67c0-1focal cephadm=17.2.7-1375-ga17c67c0-1focal ceph-mds=17.2.7-1375-ga17c67c0-1focal ceph-mgr=17.2.7-1375-ga17c67c0-1focal ceph-common=17.2.7-1375-ga17c67c0-1focal ceph-fuse=17.2.7-1375-ga17c67c0-1focal ceph-test=17.2.7-1375-ga17c67c0-1focal ceph-volume=17.2.7-1375-ga17c67c0-1focal radosgw=17.2.7-1375-ga17c67c0-1focal python3-rados=17.2.7-1375-ga17c67c0-1focal python3-rgw=17.2.7-1375-ga17c67c0-1focal python3-cephfs=17.2.7-1375-ga17c67c0-1focal python3-rbd=17.2.7-1375-ga17c67c0-1focal libcephfs2=17.2.7-1375-ga17c67c0-1focal libcephfs-dev=17.2.7-1375-ga17c67c0-1focal librados2=17.2.7-1375-ga17c67c0-1focal librbd1=17.2.7-1375-ga17c67c0-1focal rbd-fuse=17.2.7-1375-ga17c67c0-1focal python3-cephfs=17.2.7-1375-ga17c67c0-1focal cephfs-shell=17.2.7-1375-ga17c67c0-1focal cephfs-top=17.2.7-1375-ga17c67c0-1focal cephfs-mirror=17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:36:08.388 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-08-18T13:36:08.449 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-08-18T13:36:08.474 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-ga17c67c0-1focal cephadm=17.2.7-1375-ga17c67c0-1focal ceph-mds=17.2.7-1375-ga17c67c0-1focal ceph-mgr=17.2.7-1375-ga17c67c0-1focal ceph-common=17.2.7-1375-ga17c67c0-1focal ceph-fuse=17.2.7-1375-ga17c67c0-1focal ceph-test=17.2.7-1375-ga17c67c0-1focal ceph-volume=17.2.7-1375-ga17c67c0-1focal radosgw=17.2.7-1375-ga17c67c0-1focal python3-rados=17.2.7-1375-ga17c67c0-1focal python3-rgw=17.2.7-1375-ga17c67c0-1focal python3-cephfs=17.2.7-1375-ga17c67c0-1focal python3-rbd=17.2.7-1375-ga17c67c0-1focal libcephfs2=17.2.7-1375-ga17c67c0-1focal libcephfs-dev=17.2.7-1375-ga17c67c0-1focal librados2=17.2.7-1375-ga17c67c0-1focal librbd1=17.2.7-1375-ga17c67c0-1focal rbd-fuse=17.2.7-1375-ga17c67c0-1focal python3-cephfs=17.2.7-1375-ga17c67c0-1focal cephfs-shell=17.2.7-1375-ga17c67c0-1focal cephfs-top=17.2.7-1375-ga17c67c0-1focal cephfs-mirror=17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:36:08.544 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-08-18T13:36:08.602 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-08-18T13:36:08.602 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-08-18T13:36:08.734 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-08-18T13:36:08.735 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-08-18T13:36:08.790 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages were automatically installed and are no longer required: 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-18T13:36:08.792 INFO:teuthology.orchestra.run.smithi118.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-18T13:36:08.793 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-18T13:36:08.794 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout:Recommended packages: 2024-08-18T13:36:08.795 INFO:teuthology.orchestra.run.smithi118.stdout: btrfs-tools 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-18T13:36:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-18T13:36:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-18T13:36:08.916 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-18T13:36:08.917 INFO:teuthology.orchestra.run.smithi118.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-18T13:36:08.917 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-18T13:36:08.917 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-18T13:36:08.917 INFO:teuthology.orchestra.run.smithi118.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-18T13:36:08.917 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages will be upgraded: 2024-08-18T13:36:08.917 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 librbd1 2024-08-18T13:36:08.925 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-08-18T13:36:08.926 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-18T13:36:08.926 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-18T13:36:08.926 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-08-18T13:36:08.926 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-18T13:36:08.927 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-18T13:36:08.927 INFO:teuthology.orchestra.run.smithi178.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-18T13:36:08.927 INFO:teuthology.orchestra.run.smithi178.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-18T13:36:08.927 INFO:teuthology.orchestra.run.smithi178.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-18T13:36:08.927 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-18T13:36:08.928 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-18T13:36:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-18T13:36:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-18T13:36:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-18T13:36:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-18T13:36:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-18T13:36:08.930 INFO:teuthology.orchestra.run.smithi178.stdout:Recommended packages: 2024-08-18T13:36:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: btrfs-tools 2024-08-18T13:36:08.981 INFO:teuthology.orchestra.run.smithi118.stdout:2 upgraded, 114 newly installed, 0 to remove and 295 not upgraded. 2024-08-18T13:36:08.981 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 201 MB of archives. 2024-08-18T13:36:08.981 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-18T13:36:08.981 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-18T13:36:09.000 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-18T13:36:09.007 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-18T13:36:09.033 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-18T13:36:09.051 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-08-18T13:36:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-18T13:36:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-18T13:36:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-18T13:36:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-18T13:36:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-18T13:36:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-18T13:36:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-18T13:36:09.055 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages will be upgraded: 2024-08-18T13:36:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 librbd1 2024-08-18T13:36:09.063 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-18T13:36:09.065 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-18T13:36:09.074 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-18T13:36:09.077 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-18T13:36:09.095 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-18T13:36:09.096 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-18T13:36:09.101 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-18T13:36:09.118 INFO:teuthology.orchestra.run.smithi178.stdout:2 upgraded, 114 newly installed, 0 to remove and 295 not upgraded. 2024-08-18T13:36:09.118 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 201 MB of archives. 2024-08-18T13:36:09.118 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-18T13:36:09.118 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-18T13:36:09.128 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-18T13:36:09.130 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-18T13:36:09.135 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-18T13:36:09.137 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-18T13:36:09.137 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-18T13:36:09.138 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-18T13:36:09.140 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-18T13:36:09.140 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-18T13:36:09.140 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-18T13:36:09.140 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-18T13:36:09.145 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-18T13:36:09.150 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-18T13:36:09.151 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-18T13:36:09.167 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-18T13:36:09.168 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-18T13:36:09.169 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-18T13:36:09.171 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-18T13:36:09.171 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-18T13:36:09.172 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-18T13:36:09.172 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-18T13:36:09.174 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-18T13:36:09.175 INFO:teuthology.orchestra.run.smithi118.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-18T13:36:09.188 INFO:teuthology.orchestra.run.smithi118.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-18T13:36:09.189 INFO:teuthology.orchestra.run.smithi118.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-18T13:36:09.191 INFO:teuthology.orchestra.run.smithi118.stdout:Get:33 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-ga17c67c0-1focal [3,505 kB] 2024-08-18T13:36:09.201 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-18T13:36:09.203 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-18T13:36:09.205 INFO:teuthology.orchestra.run.smithi118.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-18T13:36:09.211 INFO:teuthology.orchestra.run.smithi118.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-18T13:36:09.211 INFO:teuthology.orchestra.run.smithi118.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-18T13:36:09.212 INFO:teuthology.orchestra.run.smithi118.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-18T13:36:09.212 INFO:teuthology.orchestra.run.smithi118.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-18T13:36:09.213 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-18T13:36:09.214 INFO:teuthology.orchestra.run.smithi118.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-18T13:36:09.215 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-18T13:36:09.216 INFO:teuthology.orchestra.run.smithi118.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-18T13:36:09.217 INFO:teuthology.orchestra.run.smithi118.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-18T13:36:09.222 INFO:teuthology.orchestra.run.smithi118.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-18T13:36:09.223 INFO:teuthology.orchestra.run.smithi118.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-18T13:36:09.233 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-18T13:36:09.234 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-18T13:36:09.239 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-18T13:36:09.239 INFO:teuthology.orchestra.run.smithi118.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-18T13:36:09.241 INFO:teuthology.orchestra.run.smithi118.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-18T13:36:09.241 INFO:teuthology.orchestra.run.smithi118.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-18T13:36:09.243 INFO:teuthology.orchestra.run.smithi118.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-18T13:36:09.271 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-18T13:36:09.273 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-18T13:36:09.278 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-18T13:36:09.280 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-18T13:36:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-18T13:36:09.283 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-18T13:36:09.284 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-18T13:36:09.284 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-18T13:36:09.284 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-18T13:36:09.291 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-18T13:36:09.291 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-18T13:36:09.308 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-18T13:36:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-18T13:36:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-18T13:36:09.311 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-18T13:36:09.311 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-18T13:36:09.312 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-18T13:36:09.313 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-18T13:36:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-18T13:36:09.320 INFO:teuthology.orchestra.run.smithi118.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-18T13:36:09.326 INFO:teuthology.orchestra.run.smithi178.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-18T13:36:09.328 INFO:teuthology.orchestra.run.smithi178.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-18T13:36:09.333 INFO:teuthology.orchestra.run.smithi178.stdout:Get:33 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-ga17c67c0-1focal [3,505 kB] 2024-08-18T13:36:09.335 INFO:teuthology.orchestra.run.smithi118.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-18T13:36:09.337 INFO:teuthology.orchestra.run.smithi118.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-18T13:36:09.343 INFO:teuthology.orchestra.run.smithi178.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-18T13:36:09.348 INFO:teuthology.orchestra.run.smithi118.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-18T13:36:09.348 INFO:teuthology.orchestra.run.smithi118.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-18T13:36:09.349 INFO:teuthology.orchestra.run.smithi118.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-18T13:36:09.350 INFO:teuthology.orchestra.run.smithi178.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-18T13:36:09.350 INFO:teuthology.orchestra.run.smithi178.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-18T13:36:09.351 INFO:teuthology.orchestra.run.smithi178.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-18T13:36:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-18T13:36:09.354 INFO:teuthology.orchestra.run.smithi178.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-18T13:36:09.357 INFO:teuthology.orchestra.run.smithi178.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-18T13:36:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-18T13:36:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-18T13:36:09.362 INFO:teuthology.orchestra.run.smithi178.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-18T13:36:09.378 INFO:teuthology.orchestra.run.smithi178.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-18T13:36:09.382 INFO:teuthology.orchestra.run.smithi178.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-18T13:36:09.383 INFO:teuthology.orchestra.run.smithi178.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-18T13:36:09.383 INFO:teuthology.orchestra.run.smithi178.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-18T13:36:09.415 INFO:teuthology.orchestra.run.smithi118.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-18T13:36:09.416 INFO:teuthology.orchestra.run.smithi118.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-18T13:36:09.417 INFO:teuthology.orchestra.run.smithi118.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-18T13:36:09.418 INFO:teuthology.orchestra.run.smithi118.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-18T13:36:09.420 INFO:teuthology.orchestra.run.smithi118.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-18T13:36:09.421 INFO:teuthology.orchestra.run.smithi118.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-18T13:36:09.421 INFO:teuthology.orchestra.run.smithi118.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-18T13:36:09.427 INFO:teuthology.orchestra.run.smithi118.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-18T13:36:09.432 INFO:teuthology.orchestra.run.smithi118.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-18T13:36:09.448 INFO:teuthology.orchestra.run.smithi118.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-18T13:36:09.449 INFO:teuthology.orchestra.run.smithi118.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-18T13:36:09.452 INFO:teuthology.orchestra.run.smithi118.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-18T13:36:09.455 INFO:teuthology.orchestra.run.smithi118.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-18T13:36:09.456 INFO:teuthology.orchestra.run.smithi118.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-18T13:36:09.461 INFO:teuthology.orchestra.run.smithi178.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-18T13:36:09.474 INFO:teuthology.orchestra.run.smithi118.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-18T13:36:09.476 INFO:teuthology.orchestra.run.smithi118.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-18T13:36:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-18T13:36:09.478 INFO:teuthology.orchestra.run.smithi178.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-18T13:36:09.516 INFO:teuthology.orchestra.run.smithi118.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-18T13:36:09.517 INFO:teuthology.orchestra.run.smithi118.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-18T13:36:09.517 INFO:teuthology.orchestra.run.smithi118.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-18T13:36:09.517 INFO:teuthology.orchestra.run.smithi118.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-18T13:36:09.517 INFO:teuthology.orchestra.run.smithi118.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-18T13:36:09.518 INFO:teuthology.orchestra.run.smithi118.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-18T13:36:09.529 INFO:teuthology.orchestra.run.smithi118.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-18T13:36:09.529 INFO:teuthology.orchestra.run.smithi118.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-18T13:36:09.530 INFO:teuthology.orchestra.run.smithi118.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-18T13:36:09.533 INFO:teuthology.orchestra.run.smithi118.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-18T13:36:09.550 INFO:teuthology.orchestra.run.smithi118.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-18T13:36:09.551 INFO:teuthology.orchestra.run.smithi118.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-18T13:36:09.555 INFO:teuthology.orchestra.run.smithi118.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-18T13:36:09.556 INFO:teuthology.orchestra.run.smithi118.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-18T13:36:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-18T13:36:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-18T13:36:09.617 INFO:teuthology.orchestra.run.smithi178.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-18T13:36:09.618 INFO:teuthology.orchestra.run.smithi178.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-18T13:36:09.619 INFO:teuthology.orchestra.run.smithi178.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-18T13:36:09.620 INFO:teuthology.orchestra.run.smithi178.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-18T13:36:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-18T13:36:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-18T13:36:09.623 INFO:teuthology.orchestra.run.smithi178.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-18T13:36:09.633 INFO:teuthology.orchestra.run.smithi178.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-18T13:36:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-18T13:36:09.650 INFO:teuthology.orchestra.run.smithi178.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-18T13:36:09.652 INFO:teuthology.orchestra.run.smithi178.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-18T13:36:09.653 INFO:teuthology.orchestra.run.smithi178.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-18T13:36:09.657 INFO:teuthology.orchestra.run.smithi178.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-18T13:36:09.660 INFO:teuthology.orchestra.run.smithi178.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-18T13:36:09.661 INFO:teuthology.orchestra.run.smithi178.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-18T13:36:09.675 INFO:teuthology.orchestra.run.smithi178.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-18T13:36:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-ga17c67c0-1focal [3,560 kB] 2024-08-18T13:36:09.678 INFO:teuthology.orchestra.run.smithi178.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-18T13:36:09.721 INFO:teuthology.orchestra.run.smithi178.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-18T13:36:09.722 INFO:teuthology.orchestra.run.smithi178.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-18T13:36:09.722 INFO:teuthology.orchestra.run.smithi178.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-18T13:36:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-18T13:36:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-18T13:36:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-18T13:36:09.735 INFO:teuthology.orchestra.run.smithi178.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-18T13:36:09.736 INFO:teuthology.orchestra.run.smithi178.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-18T13:36:09.736 INFO:teuthology.orchestra.run.smithi178.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-18T13:36:09.739 INFO:teuthology.orchestra.run.smithi178.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-18T13:36:09.756 INFO:teuthology.orchestra.run.smithi178.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-18T13:36:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-18T13:36:09.759 INFO:teuthology.orchestra.run.smithi178.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-18T13:36:09.760 INFO:teuthology.orchestra.run.smithi178.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-18T13:36:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-ga17c67c0-1focal [761 kB] 2024-08-18T13:36:09.782 INFO:teuthology.orchestra.run.smithi178.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-ga17c67c0-1focal [413 kB] 2024-08-18T13:36:09.788 INFO:teuthology.orchestra.run.smithi178.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-ga17c67c0-1focal [22.6 kB] 2024-08-18T13:36:09.788 INFO:teuthology.orchestra.run.smithi178.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-ga17c67c0-1focal [200 kB] 2024-08-18T13:36:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-ga17c67c0-1focal [49.1 kB] 2024-08-18T13:36:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-ga17c67c0-1focal [389 kB] 2024-08-18T13:36:09.804 INFO:teuthology.orchestra.run.smithi178.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-ga17c67c0-1focal [5,612 kB] 2024-08-18T13:36:09.910 INFO:teuthology.orchestra.run.smithi178.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-ga17c67c0-1focal [116 kB] 2024-08-18T13:36:09.911 INFO:teuthology.orchestra.run.smithi178.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-ga17c67c0-1focal [478 kB] 2024-08-18T13:36:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-ga17c67c0-1focal [26.9 MB] 2024-08-18T13:36:09.998 INFO:teuthology.orchestra.run.smithi118.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-ga17c67c0-1focal [3,560 kB] 2024-08-18T13:36:10.417 INFO:teuthology.orchestra.run.smithi118.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-ga17c67c0-1focal [761 kB] 2024-08-18T13:36:10.493 INFO:teuthology.orchestra.run.smithi118.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-ga17c67c0-1focal [413 kB] 2024-08-18T13:36:10.504 INFO:teuthology.orchestra.run.smithi118.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-ga17c67c0-1focal [22.6 kB] 2024-08-18T13:36:10.504 INFO:teuthology.orchestra.run.smithi118.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-ga17c67c0-1focal [200 kB] 2024-08-18T13:36:10.506 INFO:teuthology.orchestra.run.smithi178.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-ga17c67c0-1focal [5,174 kB] 2024-08-18T13:36:10.519 INFO:teuthology.orchestra.run.smithi118.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-ga17c67c0-1focal [49.1 kB] 2024-08-18T13:36:10.521 INFO:teuthology.orchestra.run.smithi118.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-ga17c67c0-1focal [389 kB] 2024-08-18T13:36:10.539 INFO:teuthology.orchestra.run.smithi118.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-ga17c67c0-1focal [5,612 kB] 2024-08-18T13:36:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-ga17c67c0-1focal [223 kB] 2024-08-18T13:36:10.597 INFO:teuthology.orchestra.run.smithi178.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-ga17c67c0-1focal [157 kB] 2024-08-18T13:36:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-ga17c67c0-1focal [1,594 kB] 2024-08-18T13:36:10.614 INFO:teuthology.orchestra.run.smithi178.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-ga17c67c0-1focal [4,686 kB] 2024-08-18T13:36:10.685 INFO:teuthology.orchestra.run.smithi178.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-ga17c67c0-1focal [23.1 MB] 2024-08-18T13:36:11.009 INFO:teuthology.orchestra.run.smithi118.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-ga17c67c0-1focal [116 kB] 2024-08-18T13:36:11.011 INFO:teuthology.orchestra.run.smithi118.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-ga17c67c0-1focal [478 kB] 2024-08-18T13:36:11.067 INFO:teuthology.orchestra.run.smithi118.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-ga17c67c0-1focal [26.9 MB] 2024-08-18T13:36:11.232 INFO:teuthology.orchestra.run.smithi178.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-ga17c67c0-1focal [3,948 B] 2024-08-18T13:36:11.232 INFO:teuthology.orchestra.run.smithi178.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-ga17c67c0-1focal [872 kB] 2024-08-18T13:36:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-ga17c67c0-1focal [2,416 kB] 2024-08-18T13:36:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-ga17c67c0-1focal [81.4 kB] 2024-08-18T13:36:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-ga17c67c0-1focal [120 kB] 2024-08-18T13:36:11.289 INFO:teuthology.orchestra.run.smithi178.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-ga17c67c0-1focal [6,653 kB] 2024-08-18T13:36:11.408 INFO:teuthology.orchestra.run.smithi178.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-ga17c67c0-1focal [7,888 kB] 2024-08-18T13:36:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-ga17c67c0-1focal [4,060 B] 2024-08-18T13:36:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-ga17c67c0-1focal [53.3 MB] 2024-08-18T13:36:12.455 INFO:teuthology.orchestra.run.smithi118.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-ga17c67c0-1focal [5,174 kB] 2024-08-18T13:36:12.536 INFO:teuthology.orchestra.run.smithi118.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-ga17c67c0-1focal [223 kB] 2024-08-18T13:36:12.539 INFO:teuthology.orchestra.run.smithi118.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-ga17c67c0-1focal [157 kB] 2024-08-18T13:36:12.557 INFO:teuthology.orchestra.run.smithi118.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-ga17c67c0-1focal [1,594 kB] 2024-08-18T13:36:12.607 INFO:teuthology.orchestra.run.smithi118.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-ga17c67c0-1focal [4,686 kB] 2024-08-18T13:36:12.806 INFO:teuthology.orchestra.run.smithi118.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-ga17c67c0-1focal [23.1 MB] 2024-08-18T13:36:13.420 INFO:teuthology.orchestra.run.smithi178.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-ga17c67c0-1focal [230 kB] 2024-08-18T13:36:13.424 INFO:teuthology.orchestra.run.smithi178.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-ga17c67c0-1focal [241 kB] 2024-08-18T13:36:13.428 INFO:teuthology.orchestra.run.smithi178.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-ga17c67c0-1focal [22.0 kB] 2024-08-18T13:36:13.429 INFO:teuthology.orchestra.run.smithi178.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-ga17c67c0-1focal [13.1 kB] 2024-08-18T13:36:13.430 INFO:teuthology.orchestra.run.smithi178.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-ga17c67c0-1focal [21.9 kB] 2024-08-18T13:36:13.430 INFO:teuthology.orchestra.run.smithi178.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-ga17c67c0-1focal [17.1 MB] 2024-08-18T13:36:13.480 INFO:teuthology.orchestra.run.smithi118.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-ga17c67c0-1focal [3,948 B] 2024-08-18T13:36:13.481 INFO:teuthology.orchestra.run.smithi118.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-ga17c67c0-1focal [872 kB] 2024-08-18T13:36:13.494 INFO:teuthology.orchestra.run.smithi118.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-ga17c67c0-1focal [2,416 kB] 2024-08-18T13:36:13.511 INFO:teuthology.orchestra.run.smithi118.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-ga17c67c0-1focal [81.4 kB] 2024-08-18T13:36:13.512 INFO:teuthology.orchestra.run.smithi118.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-ga17c67c0-1focal [120 kB] 2024-08-18T13:36:13.513 INFO:teuthology.orchestra.run.smithi118.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-ga17c67c0-1focal [6,653 kB] 2024-08-18T13:36:13.703 INFO:teuthology.orchestra.run.smithi118.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-ga17c67c0-1focal [7,888 kB] 2024-08-18T13:36:13.957 INFO:teuthology.orchestra.run.smithi178.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-ga17c67c0-1focal [80.6 kB] 2024-08-18T13:36:13.988 INFO:teuthology.orchestra.run.smithi118.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-ga17c67c0-1focal [4,060 B] 2024-08-18T13:36:13.988 INFO:teuthology.orchestra.run.smithi118.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-ga17c67c0-1focal [53.3 MB] 2024-08-18T13:36:14.607 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 201 MB in 5s (41.1 MB/s) 2024-08-18T13:36:14.753 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-18T13:36:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-18T13:36:14.811 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-18T13:36:14.872 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-18T13:36:15.136 INFO:teuthology.orchestra.run.smithi118.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-ga17c67c0-1focal [230 kB] 2024-08-18T13:36:15.142 INFO:teuthology.orchestra.run.smithi118.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-ga17c67c0-1focal [241 kB] 2024-08-18T13:36:15.146 INFO:teuthology.orchestra.run.smithi118.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-ga17c67c0-1focal [22.0 kB] 2024-08-18T13:36:15.147 INFO:teuthology.orchestra.run.smithi118.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-ga17c67c0-1focal [13.1 kB] 2024-08-18T13:36:15.147 INFO:teuthology.orchestra.run.smithi118.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-ga17c67c0-1focal [21.9 kB] 2024-08-18T13:36:15.148 INFO:teuthology.orchestra.run.smithi118.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-ga17c67c0-1focal [17.1 MB] 2024-08-18T13:36:15.213 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-18T13:36:15.233 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-18T13:36:15.271 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-18T13:36:15.457 INFO:teuthology.orchestra.run.smithi118.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/a17c67c052c1476b20f99d847823755b26703128/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-ga17c67c0-1focal [80.6 kB] 2024-08-18T13:36:15.574 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cmd2. 2024-08-18T13:36:15.590 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-18T13:36:15.623 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-18T13:36:15.985 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-18T13:36:16.002 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-18T13:36:16.085 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-18T13:36:16.097 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 201 MB in 7s (30.8 MB/s) 2024-08-18T13:36:16.224 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-18T13:36:16.280 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-18T13:36:16.285 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-18T13:36:16.355 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-18T13:36:16.466 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:16.589 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librbd1 (17.2.7-1375-ga17c67c0-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-18T13:36:16.658 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-18T13:36:16.674 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-18T13:36:16.750 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-18T13:36:17.053 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cmd2. 2024-08-18T13:36:17.069 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-18T13:36:17.136 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-18T13:36:17.321 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:17.452 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librados2 (17.2.7-1375-ga17c67c0-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-18T13:36:17.498 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-18T13:36:17.513 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-18T13:36:17.623 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-18T13:36:17.993 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:18.105 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcephfs2. 2024-08-18T13:36:18.119 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librbd1 (17.2.7-1375-ga17c67c0-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-18T13:36:18.125 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:18.163 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcephfs2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:18.549 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rados. 2024-08-18T13:36:18.568 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:18.624 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rados (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:18.833 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:18.943 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librados2 (17.2.7-1375-ga17c67c0-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-18T13:36:18.994 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-18T13:36:19.013 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:19.052 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:19.279 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cephfs. 2024-08-18T13:36:19.296 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:19.329 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cephfs (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:19.630 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs2. 2024-08-18T13:36:19.646 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:19.688 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:19.715 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-18T13:36:19.731 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:19.806 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:20.041 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rados. 2024-08-18T13:36:20.057 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:20.099 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rados (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:20.160 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-prettytable. 2024-08-18T13:36:20.177 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-18T13:36:20.218 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-18T13:36:20.477 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-18T13:36:20.487 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rbd. 2024-08-18T13:36:20.493 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:20.505 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:20.535 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:20.545 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rbd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:20.829 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cephfs. 2024-08-18T13:36:20.848 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:20.888 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cephfs (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:20.898 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-18T13:36:20.914 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-18T13:36:20.948 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-18T13:36:21.216 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-18T13:36:21.235 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:21.265 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:21.351 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-18T13:36:21.369 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-18T13:36:21.409 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-18T13:36:21.619 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-prettytable. 2024-08-18T13:36:21.636 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-18T13:36:21.677 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-18T13:36:21.770 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-18T13:36:21.787 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-18T13:36:21.828 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-18T13:36:21.971 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rbd. 2024-08-18T13:36:21.989 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:22.030 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rbd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:22.181 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-18T13:36:22.201 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-18T13:36:22.239 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-18T13:36:22.399 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-18T13:36:22.419 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-18T13:36:22.457 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-18T13:36:22.551 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-18T13:36:22.569 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-18T13:36:22.626 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-18T13:36:22.827 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-18T13:36:22.844 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-18T13:36:22.885 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-18T13:36:23.121 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-18T13:36:23.136 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-18T13:36:23.214 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:36:23.255 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-18T13:36:23.272 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-18T13:36:23.313 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-18T13:36:23.692 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-18T13:36:23.708 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-18T13:36:23.723 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-18T13:36:23.742 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-18T13:36:23.775 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-18T13:36:23.807 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:36:24.078 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-18T13:36:24.094 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-18T13:36:24.170 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-18T13:36:24.185 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-18T13:36:24.202 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-18T13:36:24.243 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:36:24.656 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-18T13:36:24.663 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-18T13:36:24.675 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-18T13:36:24.679 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-18T13:36:24.725 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:36:24.738 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-18T13:36:25.124 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librgw2. 2024-08-18T13:36:25.142 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:25.182 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librgw2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:25.235 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-18T13:36:25.250 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-18T13:36:25.310 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:36:25.696 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-18T13:36:25.715 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-18T13:36:25.754 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:36:26.193 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rgw. 2024-08-18T13:36:26.213 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:26.251 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:26.291 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-18T13:36:26.311 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-18T13:36:26.349 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-18T13:36:26.621 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liboath0. 2024-08-18T13:36:26.636 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-18T13:36:26.686 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package librgw2. 2024-08-18T13:36:26.704 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:26.712 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-18T13:36:26.736 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking librgw2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:27.056 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libradosstriper1. 2024-08-18T13:36:27.072 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:27.115 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libradosstriper1 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:27.530 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rgw. 2024-08-18T13:36:27.548 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:27.588 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:27.635 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-common. 2024-08-18T13:36:27.654 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:27.693 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:27.983 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liboath0. 2024-08-18T13:36:27.998 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-18T13:36:28.041 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-18T13:36:28.360 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libradosstriper1. 2024-08-18T13:36:28.377 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:28.410 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libradosstriper1 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:28.889 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-common. 2024-08-18T13:36:28.905 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:28.947 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:30.126 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-base. 2024-08-18T13:36:30.142 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:30.192 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-base (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:31.054 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cryptography. 2024-08-18T13:36:31.073 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-18T13:36:31.104 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-18T13:36:31.491 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-openssl. 2024-08-18T13:36:31.509 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-18T13:36:31.527 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-base. 2024-08-18T13:36:31.545 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:31.549 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-18T13:36:31.616 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-base (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:31.919 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-packaging. 2024-08-18T13:36:31.935 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-18T13:36:31.977 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-18T13:36:32.263 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-18T13:36:32.279 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:32.321 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:32.524 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cryptography. 2024-08-18T13:36:32.541 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-18T13:36:32.582 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-18T13:36:32.734 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-18T13:36:32.751 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-18T13:36:32.792 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-18T13:36:32.986 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-openssl. 2024-08-18T13:36:33.004 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-18T13:36:33.044 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-18T13:36:33.078 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-18T13:36:33.097 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-18T13:36:33.137 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-18T13:36:33.397 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-packaging. 2024-08-18T13:36:33.413 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-18T13:36:33.455 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-18T13:36:33.456 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-natsort. 2024-08-18T13:36:33.475 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-18T13:36:33.506 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-18T13:36:33.708 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-18T13:36:33.725 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:33.758 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:33.826 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jwt. 2024-08-18T13:36:33.842 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-18T13:36:33.884 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-18T13:36:34.155 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-18T13:36:34.170 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-18T13:36:34.213 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-18T13:36:34.254 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-logutils. 2024-08-18T13:36:34.274 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-18T13:36:34.312 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-18T13:36:34.499 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-18T13:36:34.516 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-18T13:36:34.549 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-18T13:36:34.640 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-18T13:36:34.656 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-18T13:36:34.698 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-18T13:36:34.861 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-natsort. 2024-08-18T13:36:34.879 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-18T13:36:34.911 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-18T13:36:35.009 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-mako. 2024-08-18T13:36:35.029 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-18T13:36:35.067 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-18T13:36:35.192 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jwt. 2024-08-18T13:36:35.211 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-18T13:36:35.256 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-18T13:36:35.421 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-18T13:36:35.437 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-18T13:36:35.512 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-18T13:36:35.609 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-logutils. 2024-08-18T13:36:35.628 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-18T13:36:35.659 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-18T13:36:35.840 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-18T13:36:35.858 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-18T13:36:35.898 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-18T13:36:35.987 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-18T13:36:36.004 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-18T13:36:36.045 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-18T13:36:36.251 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-webob. 2024-08-18T13:36:36.267 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-18T13:36:36.309 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-18T13:36:36.398 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-mako. 2024-08-18T13:36:36.417 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-18T13:36:36.456 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-18T13:36:36.654 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-18T13:36:36.674 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-18T13:36:36.737 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-18T13:36:36.802 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-18T13:36:36.820 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-18T13:36:36.860 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-18T13:36:37.073 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-bs4. 2024-08-18T13:36:37.091 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-18T13:36:37.131 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-18T13:36:37.179 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-18T13:36:37.197 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-18T13:36:37.238 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-18T13:36:37.491 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webob. 2024-08-18T13:36:37.510 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-18T13:36:37.517 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-waitress. 2024-08-18T13:36:37.534 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-18T13:36:37.579 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-18T13:36:37.582 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-18T13:36:37.910 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-18T13:36:37.930 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-18T13:36:37.963 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-tempita. 2024-08-18T13:36:37.977 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-18T13:36:37.981 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-18T13:36:38.029 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-18T13:36:38.280 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-bs4. 2024-08-18T13:36:38.297 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-18T13:36:38.323 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-paste. 2024-08-18T13:36:38.340 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-18T13:36:38.354 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-18T13:36:38.373 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-18T13:36:38.710 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-18T13:36:38.725 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-18T13:36:38.733 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-waitress. 2024-08-18T13:36:38.750 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-18T13:36:38.768 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-18T13:36:38.802 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-18T13:36:39.104 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-18T13:36:39.122 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-18T13:36:39.136 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-tempita. 2024-08-18T13:36:39.155 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-18T13:36:39.162 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-18T13:36:39.194 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-18T13:36:39.512 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-paste. 2024-08-18T13:36:39.515 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-webtest. 2024-08-18T13:36:39.531 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-18T13:36:39.534 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-18T13:36:39.572 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-18T13:36:39.579 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-18T13:36:39.899 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pecan. 2024-08-18T13:36:39.917 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-18T13:36:39.917 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-18T13:36:39.934 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-18T13:36:39.957 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-18T13:36:39.966 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-18T13:36:40.311 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-18T13:36:40.327 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-18T13:36:40.337 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-18T13:36:40.356 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-18T13:36:40.369 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-18T13:36:40.395 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-18T13:36:40.656 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webtest. 2024-08-18T13:36:40.675 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-18T13:36:40.739 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-18T13:36:40.765 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-18T13:36:40.782 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:40.839 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:41.050 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pecan. 2024-08-18T13:36:41.066 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-18T13:36:41.192 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr. 2024-08-18T13:36:41.198 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-18T13:36:41.210 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:41.352 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:42.247 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-18T13:36:42.267 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-18T13:36:42.331 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-18T13:36:42.496 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mon. 2024-08-18T13:36:42.512 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:42.554 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mon (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:42.717 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-18T13:36:42.736 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:42.775 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:43.156 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr. 2024-08-18T13:36:43.173 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:43.214 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:43.357 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-osd. 2024-08-18T13:36:43.373 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:43.424 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-osd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:43.761 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mon. 2024-08-18T13:36:43.777 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:43.827 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mon (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:44.739 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-osd. 2024-08-18T13:36:44.757 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:44.797 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-osd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:45.548 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph. 2024-08-18T13:36:45.563 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:45.605 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:45.950 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-fuse. 2024-08-18T13:36:45.967 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:46.024 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:46.435 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mds. 2024-08-18T13:36:46.454 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:46.493 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mds (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:46.975 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph. 2024-08-18T13:36:46.992 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:47.030 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package cephadm. 2024-08-18T13:36:47.047 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:47.087 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:47.125 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:47.453 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-fuse. 2024-08-18T13:36:47.465 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jinja2. 2024-08-18T13:36:47.469 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:47.485 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-18T13:36:47.553 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:47.557 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-18T13:36:47.918 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-18T13:36:47.935 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:47.976 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:47.989 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mds. 2024-08-18T13:36:48.007 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:48.047 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mds (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:48.397 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-18T13:36:48.417 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-18T13:36:48.472 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-18T13:36:48.618 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephadm. 2024-08-18T13:36:48.634 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:36:48.684 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:48.716 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjs-underscore. 2024-08-18T13:36:48.732 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-18T13:36:48.765 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-18T13:36:49.035 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-18T13:36:49.045 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-jinja2. 2024-08-18T13:36:49.051 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-18T13:36:49.061 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-18T13:36:49.084 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-18T13:36:49.137 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-18T13:36:49.412 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-routes. 2024-08-18T13:36:49.431 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-18T13:36:49.456 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-18T13:36:49.470 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-18T13:36:49.473 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:49.548 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:49.824 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-18T13:36:49.840 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:49.881 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:49.935 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-18T13:36:49.954 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-18T13:36:49.993 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-18T13:36:50.212 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjs-underscore. 2024-08-18T13:36:50.229 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-18T13:36:50.298 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-18T13:36:50.573 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-18T13:36:50.589 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-18T13:36:50.631 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-18T13:36:50.976 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-routes. 2024-08-18T13:36:50.991 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-18T13:36:51.034 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-18T13:36:51.329 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-decorator. 2024-08-18T13:36:51.349 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-18T13:36:51.370 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-18T13:36:51.390 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:51.420 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-18T13:36:51.450 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:51.731 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-18T13:36:51.750 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-18T13:36:51.789 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-18T13:36:52.117 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-scipy. 2024-08-18T13:36:52.137 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-18T13:36:52.175 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-18T13:36:52.892 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-decorator. 2024-08-18T13:36:52.909 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-18T13:36:52.950 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-18T13:36:53.279 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-18T13:36:53.298 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-18T13:36:53.328 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-18T13:36:53.550 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-18T13:36:53.570 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-18T13:36:53.608 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-18T13:36:53.664 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-scipy. 2024-08-18T13:36:53.681 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-18T13:36:53.722 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-18T13:36:54.188 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-joblib. 2024-08-18T13:36:54.204 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-18T13:36:54.246 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-18T13:36:54.608 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-sklearn. 2024-08-18T13:36:54.626 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-18T13:36:54.666 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-18T13:36:55.100 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-18T13:36:55.117 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-18T13:36:55.166 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-18T13:36:55.441 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-18T13:36:55.457 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:55.499 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:55.746 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-joblib. 2024-08-18T13:36:55.762 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-18T13:36:55.809 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-18T13:36:56.157 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-sklearn. 2024-08-18T13:36:56.176 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-18T13:36:56.207 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-18T13:36:56.677 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cachetools. 2024-08-18T13:36:56.693 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-18T13:36:56.735 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-18T13:36:56.968 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-18T13:36:56.985 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:36:57.025 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:36:57.105 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-18T13:36:57.125 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-18T13:36:57.163 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-18T13:36:57.908 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-18T13:36:57.924 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-18T13:36:57.966 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-18T13:36:58.285 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rsa. 2024-08-18T13:36:58.305 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-18T13:36:58.329 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-cachetools. 2024-08-18T13:36:58.347 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-18T13:36:58.360 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-18T13:36:58.387 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-18T13:36:58.705 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-google-auth. 2024-08-18T13:36:58.715 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-18T13:36:58.721 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-18T13:36:58.732 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-18T13:36:58.773 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-18T13:36:58.796 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-18T13:36:59.135 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-18T13:36:59.149 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-blinker. 2024-08-18T13:36:59.152 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-18T13:36:59.167 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-18T13:36:59.193 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-18T13:36:59.207 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-18T13:36:59.538 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-rsa. 2024-08-18T13:36:59.543 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-18T13:36:59.556 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-18T13:36:59.560 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-18T13:36:59.596 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-18T13:36:59.601 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-18T13:36:59.924 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-google-auth. 2024-08-18T13:36:59.943 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-18T13:36:59.982 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-18T13:37:00.091 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-18T13:37:00.107 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-18T13:37:00.149 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-18T13:37:00.311 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-blinker. 2024-08-18T13:37:00.327 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-18T13:37:00.369 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-18T13:37:00.477 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-websocket. 2024-08-18T13:37:00.497 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-18T13:37:00.560 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-18T13:37:00.705 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-18T13:37:00.721 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-18T13:37:00.763 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-18T13:37:00.881 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-18T13:37:00.902 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-18T13:37:00.939 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-18T13:37:01.195 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-18T13:37:01.213 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-18T13:37:01.253 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-18T13:37:01.544 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-18T13:37:01.557 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-websocket. 2024-08-18T13:37:01.560 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:01.574 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-18T13:37:01.602 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:01.615 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-18T13:37:01.952 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-18T13:37:01.964 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-18T13:37:01.969 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-18T13:37:01.981 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-18T13:37:02.010 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-18T13:37:02.022 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-18T13:37:02.442 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-18T13:37:02.462 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-18T13:37:02.500 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:02.592 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-18T13:37:02.611 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:02.650 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:02.803 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package jq. 2024-08-18T13:37:02.820 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-18T13:37:02.861 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:02.986 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-18T13:37:03.004 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-18T13:37:03.044 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-18T13:37:03.141 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package socat. 2024-08-18T13:37:03.157 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-18T13:37:03.200 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-18T13:37:03.418 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-18T13:37:03.436 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-18T13:37:03.476 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:03.521 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package xmlstarlet. 2024-08-18T13:37:03.539 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-18T13:37:03.579 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-18T13:37:03.737 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package jq. 2024-08-18T13:37:03.754 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-18T13:37:03.796 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:03.950 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-test. 2024-08-18T13:37:03.967 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:04.033 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-test (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:04.057 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package socat. 2024-08-18T13:37:04.074 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-18T13:37:04.132 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-18T13:37:04.493 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package xmlstarlet. 2024-08-18T13:37:04.511 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-18T13:37:04.543 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-18T13:37:04.898 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-test. 2024-08-18T13:37:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:04.956 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-test (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:08.763 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-volume. 2024-08-18T13:37:08.782 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:08.830 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-volume (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:09.261 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-18T13:37:09.277 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:09.343 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking cephfs-mirror (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-colorama. 2024-08-18T13:37:09.742 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-18T13:37:09.777 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ceph-volume. 2024-08-18T13:37:09.780 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-18T13:37:09.795 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:09.835 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ceph-volume (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:10.080 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package cephfs-shell. 2024-08-18T13:37:10.099 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:10.138 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking cephfs-shell (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:10.295 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-18T13:37:10.313 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:10.353 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephfs-mirror (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package cephfs-top. 2024-08-18T13:37:10.409 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:10.450 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking cephfs-top (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:10.707 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-colorama. 2024-08-18T13:37:10.725 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-18T13:37:10.737 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-18T13:37:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-18T13:37:10.790 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-18T13:37:10.795 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:11.068 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephfs-shell. 2024-08-18T13:37:11.085 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:11.126 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephfs-shell (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:11.359 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-18T13:37:11.378 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:11.382 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package cephfs-top. 2024-08-18T13:37:11.398 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-ga17c67c0-1focal_all.deb ... 2024-08-18T13:37:11.417 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcephfs-dev (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:11.440 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking cephfs-top (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:11.717 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-18T13:37:11.734 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-18T13:37:11.775 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:11.862 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package nvme-cli. 2024-08-18T13:37:11.882 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-18T13:37:11.937 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-18T13:37:12.292 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-18T13:37:12.311 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-18T13:37:12.338 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-18T13:37:12.350 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:12.356 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:12.396 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libcephfs-dev (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:12.799 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package nvme-cli. 2024-08-18T13:37:12.821 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-18T13:37:12.875 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-18T13:37:13.221 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-18T13:37:13.238 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-18T13:37:13.270 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:14.060 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-18T13:37:14.080 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-18T13:37:14.118 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-18T13:37:14.479 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-18T13:37:14.497 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-18T13:37:14.537 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-18T13:37:14.874 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-attr. 2024-08-18T13:37:14.894 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-18T13:37:14.940 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-18T13:37:15.028 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-18T13:37:15.049 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-18T13:37:15.085 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-18T13:37:15.285 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-webencodings. 2024-08-18T13:37:15.301 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-18T13:37:15.343 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-18T13:37:15.430 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-18T13:37:15.449 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-18T13:37:15.511 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-18T13:37:15.671 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-html5lib. 2024-08-18T13:37:15.688 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-18T13:37:15.729 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-18T13:37:15.833 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-attr. 2024-08-18T13:37:15.850 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-18T13:37:15.916 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-18T13:37:16.099 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-18T13:37:16.116 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-18T13:37:16.182 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-18T13:37:16.236 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-webencodings. 2024-08-18T13:37:16.256 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-18T13:37:16.319 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-18T13:37:16.605 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-html5lib. 2024-08-18T13:37:16.623 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-18T13:37:16.678 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pastescript. 2024-08-18T13:37:16.688 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-18T13:37:16.698 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-18T13:37:16.736 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-18T13:37:17.008 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-18T13:37:17.028 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-18T13:37:17.091 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-18T13:37:17.114 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pluggy. 2024-08-18T13:37:17.131 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-18T13:37:17.172 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-18T13:37:17.509 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-psutil. 2024-08-18T13:37:17.525 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-18T13:37:17.567 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-18T13:37:17.611 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pastescript. 2024-08-18T13:37:17.631 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-18T13:37:17.686 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-18T13:37:17.929 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-py. 2024-08-18T13:37:17.945 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-18T13:37:17.986 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-18T13:37:18.064 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pluggy. 2024-08-18T13:37:18.084 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-18T13:37:18.122 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-18T13:37:18.390 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-18T13:37:18.407 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-18T13:37:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-18T13:37:18.467 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-psutil. 2024-08-18T13:37:18.488 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-18T13:37:18.525 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-18T13:37:18.802 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pytest. 2024-08-18T13:37:18.822 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-18T13:37:18.859 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-18T13:37:18.887 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-py. 2024-08-18T13:37:18.908 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-18T13:37:18.945 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-18T13:37:19.221 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-simplejson. 2024-08-18T13:37:19.239 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-18T13:37:19.304 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-18T13:37:19.323 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-18T13:37:19.341 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-18T13:37:19.406 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-18T13:37:19.582 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-18T13:37:19.599 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-18T13:37:19.657 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-18T13:37:19.718 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-pytest. 2024-08-18T13:37:19.734 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-18T13:37:19.802 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-18T13:37:20.180 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package python3-simplejson. 2024-08-18T13:37:20.200 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-18T13:37:20.246 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-18T13:37:20.462 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package radosgw. 2024-08-18T13:37:20.480 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:20.519 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking radosgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:20.532 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-18T13:37:20.549 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-18T13:37:20.616 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-18T13:37:21.373 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package radosgw. 2024-08-18T13:37:21.390 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:21.430 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking radosgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:22.302 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package rbd-fuse. 2024-08-18T13:37:22.322 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:22.368 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking rbd-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:22.657 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-18T13:37:22.973 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-18T13:37:23.091 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-18T13:37:23.217 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-18T13:37:23.276 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package rbd-fuse. 2024-08-18T13:37:23.293 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-ga17c67c0-1focal_amd64.deb ... 2024-08-18T13:37:23.334 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking rbd-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:23.435 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:23.664 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-18T13:37:23.964 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-18T13:37:24.031 INFO:teuthology.orchestra.run.smithi178.stdout:Adding system user cephadm....done 2024-08-18T13:37:24.090 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-18T13:37:24.227 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:24.249 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-18T13:37:24.354 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-18T13:37:24.453 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:24.706 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-18T13:37:25.007 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:25.026 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user cephadm....done 2024-08-18T13:37:25.150 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-18T13:37:25.204 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:25.276 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-18T13:37:25.330 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-18T13:37:25.619 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-18T13:37:25.698 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-18T13:37:25.983 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-18T13:37:26.063 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-18T13:37:26.109 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-18T13:37:26.254 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-18T13:37:26.356 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-18T13:37:26.614 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-18T13:37:26.651 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-18T13:37:26.960 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-18T13:37:27.016 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-18T13:37:27.261 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-18T13:37:27.317 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-18T13:37:27.596 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-ceph-argparse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:27.635 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-18T13:37:27.905 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-18T13:37:27.962 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-18T13:37:28.015 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-18T13:37:28.263 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-18T13:37:28.291 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-18T13:37:28.934 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-18T13:37:28.964 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-argparse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:29.260 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-18T13:37:29.323 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-18T13:37:29.386 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-18T13:37:29.483 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-18T13:37:29.755 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-18T13:37:29.801 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-18T13:37:30.081 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-18T13:37:30.094 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-18T13:37:30.390 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-18T13:37:30.412 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-18T13:37:30.513 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-18T13:37:30.775 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-18T13:37:30.922 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-18T13:37:31.102 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-18T13:37:31.241 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-18T13:37:31.444 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-18T13:37:31.542 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-18T13:37:31.737 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-18T13:37:31.877 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-18T13:37:32.046 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-18T13:37:32.173 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-18T13:37:32.194 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-18T13:37:32.307 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-18T13:37:32.546 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-18T13:37:32.609 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-18T13:37:32.864 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-18T13:37:33.010 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-18T13:37:33.165 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-18T13:37:33.320 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-18T13:37:33.324 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-18T13:37:33.450 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-18T13:37:33.680 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-18T13:37:33.752 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-18T13:37:34.107 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-18T13:37:34.179 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-18T13:37:34.208 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:34.510 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:37:34.538 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-18T13:37:34.677 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-18T13:37:34.898 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-18T13:37:35.020 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-18T13:37:35.147 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-18T13:37:35.258 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-18T13:37:35.289 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-18T13:37:35.410 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:35.584 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-18T13:37:35.745 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:37:35.871 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-18T13:37:35.903 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:37:36.035 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-18T13:37:36.161 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-18T13:37:36.191 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-18T13:37:36.307 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-18T13:37:36.475 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-18T13:37:36.530 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-18T13:37:36.759 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-18T13:37:36.830 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-18T13:37:37.101 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-18T13:37:37.111 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:37:37.260 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-18T13:37:37.262 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-18T13:37:37.363 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-18T13:37:37.536 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-18T13:37:37.673 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-18T13:37:37.837 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-18T13:37:37.979 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-18T13:37:38.138 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-18T13:37:38.275 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-18T13:37:38.435 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-18T13:37:38.520 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-18T13:37:38.720 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-18T13:37:38.774 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-18T13:37:38.900 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-18T13:37:39.054 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-18T13:37:39.251 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-18T13:37:39.347 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-18T13:37:39.644 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-18T13:37:39.684 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-18T13:37:39.770 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-18T13:37:39.896 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-18T13:37:39.967 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-18T13:37:40.093 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-18T13:37:40.264 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-18T13:37:40.419 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-18T13:37:40.557 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librados2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:40.683 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-18T13:37:40.788 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-18T13:37:40.889 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-18T13:37:40.998 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-18T13:37:41.325 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-18T13:37:41.609 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librados2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:41.744 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-18T13:37:42.147 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:42.273 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:42.432 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-18T13:37:42.809 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-18T13:37:43.118 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-18T13:37:43.208 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:43.334 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:43.428 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:37:43.494 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-18T13:37:43.579 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-18T13:37:43.871 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-18T13:37:43.913 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcephfs2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:44.056 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-18T13:37:44.155 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-18T13:37:44.441 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libradosstriper1 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:44.482 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-18T13:37:44.575 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-18T13:37:44.641 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-18T13:37:44.927 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-18T13:37:45.009 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:45.144 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-18T13:37:45.479 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libradosstriper1 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:45.580 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-18T13:37:45.907 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-18T13:37:46.015 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librbd1 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:46.158 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-18T13:37:46.317 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-18T13:37:46.443 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:46.594 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-18T13:37:46.895 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-18T13:37:46.996 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librbd1 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:47.021 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-18T13:37:47.097 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-18T13:37:47.198 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-18T13:37:47.366 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-18T13:37:47.433 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:47.492 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-18T13:37:47.584 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:47.794 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-18T13:37:47.805 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-18T13:37:47.806 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-18T13:37:47.920 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-18T13:37:48.252 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcephfs-dev (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:48.323 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:48.403 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-18T13:37:48.470 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:48.698 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-18T13:37:48.699 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-18T13:37:48.704 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-18T13:37:49.031 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-18T13:37:49.134 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libcephfs-dev (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:49.302 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-18T13:37:49.417 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rados (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:49.543 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librgw2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:49.596 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-18T13:37:49.694 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up cephfs-top (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:49.820 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rbd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:49.930 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-18T13:37:49.979 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up rbd-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:50.105 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-18T13:37:50.292 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rados (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:50.418 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up librgw2 (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:50.423 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-18T13:37:50.611 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephfs-top (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:50.729 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rbd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:50.758 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:50.888 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up rbd-fuse (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:50.917 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cephfs (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:51.014 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-18T13:37:51.043 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up cephfs-shell (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:51.194 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-18T13:37:51.332 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-18T13:37:51.659 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-rgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:51.784 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-cephfs (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:51.911 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephfs-shell (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:52.053 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-18T13:37:52.114 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:52.717 INFO:teuthology.orchestra.run.smithi178.stdout:Adding group ceph....done 2024-08-18T13:37:52.982 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-common (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:53.144 INFO:teuthology.orchestra.run.smithi178.stdout:Adding system user ceph....done 2024-08-18T13:37:53.353 INFO:teuthology.orchestra.run.smithi178.stdout:Setting system user ceph properties....done 2024-08-18T13:37:53.378 INFO:teuthology.orchestra.run.smithi178.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-18T13:37:53.522 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-18T13:37:53.552 INFO:teuthology.orchestra.run.smithi118.stdout:Adding group ceph....done 2024-08-18T13:37:53.832 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-18T13:37:53.937 INFO:teuthology.orchestra.run.smithi118.stdout:Adding system user ceph....done 2024-08-18T13:37:54.147 INFO:teuthology.orchestra.run.smithi118.stdout:Setting system user ceph properties....done 2024-08-18T13:37:54.179 INFO:teuthology.orchestra.run.smithi118.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-18T13:37:54.299 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up cephfs-mirror (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:54.318 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-18T13:37:54.521 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-18T13:37:54.521 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-18T13:37:54.626 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-18T13:37:54.959 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-test (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:55.076 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up cephfs-mirror (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:55.110 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up radosgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:55.335 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-18T13:37:55.336 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-18T13:37:55.383 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-18T13:37:55.383 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-18T13:37:55.811 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-test (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:55.957 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up radosgw (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:56.045 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-base (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:56.238 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-18T13:37:56.238 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-18T13:37:56.418 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-18T13:37:56.863 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mds (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:56.881 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-base (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:57.085 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-18T13:37:57.085 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-18T13:37:57.320 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-18T13:37:57.515 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:57.746 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-18T13:37:57.746 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-18T13:37:57.889 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mds (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:58.179 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-18T13:37:58.179 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-18T13:37:58.199 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-osd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:58.611 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:58.715 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-18T13:37:58.715 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-18T13:37:58.868 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-18T13:37:58.868 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-18T13:37:59.143 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:59.294 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:59.304 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-osd (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:37:59.790 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-18T13:37:59.791 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-18T13:37:59.860 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mon (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:00.142 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-18T13:38:00.143 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-18T13:38:00.299 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:00.424 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:00.589 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:00.609 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mon (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:00.715 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:00.867 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-18T13:38:00.867 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-18T13:38:00.891 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:01.034 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-volume (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:01.186 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-18T13:38:01.311 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:01.354 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-18T13:38:01.453 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:01.555 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:01.647 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-18T13:38:01.689 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ceph-volume (17.2.7-1375-ga17c67c0-1focal) ... 2024-08-18T13:38:01.849 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-18T13:38:02.001 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-18T13:38:02.286 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-18T13:38:04.583 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-18T13:38:05.114 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-18T13:38:05.333 INFO:teuthology.orchestra.run.smithi178.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-18T13:38:05.336 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-18T13:38:05.407 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-08-18T13:38:05.624 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-08-18T13:38:05.624 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-08-18T13:38:05.731 INFO:teuthology.orchestra.run.smithi118.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-18T13:38:05.734 DEBUG:teuthology.orchestra.run.smithi118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-18T13:38:05.811 INFO:teuthology.orchestra.run.smithi118.stdout:Reading package lists... 2024-08-18T13:38:05.840 INFO:teuthology.orchestra.run.smithi178.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-18T13:38:05.840 INFO:teuthology.orchestra.run.smithi178.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-18T13:38:05.840 INFO:teuthology.orchestra.run.smithi178.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-18T13:38:05.840 INFO:teuthology.orchestra.run.smithi178.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-18T13:38:05.840 INFO:teuthology.orchestra.run.smithi178.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-18T13:38:05.840 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-08-18T13:38:05.841 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-18T13:38:05.841 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-18T13:38:05.841 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-08-18T13:38:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-18T13:38:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-18T13:38:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-18T13:38:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: usb-modeswitch-data wpasupplicant 2024-08-18T13:38:05.843 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-08-18T13:38:05.843 INFO:teuthology.orchestra.run.smithi178.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-18T13:38:05.844 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-18T13:38:05.893 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-08-18T13:38:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-18T13:38:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-18T13:38:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-18T13:38:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-18T13:38:05.895 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages will be upgraded: 2024-08-18T13:38:05.896 INFO:teuthology.orchestra.run.smithi178.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-18T13:38:05.952 INFO:teuthology.orchestra.run.smithi178.stdout:6 upgraded, 23 newly installed, 0 to remove and 289 not upgraded. 2024-08-18T13:38:05.953 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 10.4 MB of archives. 2024-08-18T13:38:05.953 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-18T13:38:05.953 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-18T13:38:06.028 INFO:teuthology.orchestra.run.smithi118.stdout:Building dependency tree... 2024-08-18T13:38:06.029 INFO:teuthology.orchestra.run.smithi118.stdout:Reading state information... 2024-08-18T13:38:06.085 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-18T13:38:06.086 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-18T13:38:06.087 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-18T13:38:06.093 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-18T13:38:06.095 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-18T13:38:06.109 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-18T13:38:06.110 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-18T13:38:06.111 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-18T13:38:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-18T13:38:06.112 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-18T13:38:06.118 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-18T13:38:06.119 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-18T13:38:06.121 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-18T13:38:06.121 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-18T13:38:06.135 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-18T13:38:06.135 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-18T13:38:06.141 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-18T13:38:06.148 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-18T13:38:06.148 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-18T13:38:06.162 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-18T13:38:06.162 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-18T13:38:06.170 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-18T13:38:06.180 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-18T13:38:06.195 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-18T13:38:06.198 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-18T13:38:06.198 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-18T13:38:06.198 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-18T13:38:06.199 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-18T13:38:06.253 INFO:teuthology.orchestra.run.smithi118.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-18T13:38:06.253 INFO:teuthology.orchestra.run.smithi118.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-18T13:38:06.253 INFO:teuthology.orchestra.run.smithi118.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-18T13:38:06.253 INFO:teuthology.orchestra.run.smithi118.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-18T13:38:06.253 INFO:teuthology.orchestra.run.smithi118.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-18T13:38:06.253 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages were automatically installed and are no longer required: 2024-08-18T13:38:06.254 INFO:teuthology.orchestra.run.smithi118.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-18T13:38:06.254 INFO:teuthology.orchestra.run.smithi118.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-18T13:38:06.255 INFO:teuthology.orchestra.run.smithi118.stdout:The following additional packages will be installed: 2024-08-18T13:38:06.255 INFO:teuthology.orchestra.run.smithi118.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-18T13:38:06.255 INFO:teuthology.orchestra.run.smithi118.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-18T13:38:06.255 INFO:teuthology.orchestra.run.smithi118.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-18T13:38:06.256 INFO:teuthology.orchestra.run.smithi118.stdout: usb-modeswitch-data wpasupplicant 2024-08-18T13:38:06.257 INFO:teuthology.orchestra.run.smithi118.stdout:Suggested packages: 2024-08-18T13:38:06.257 INFO:teuthology.orchestra.run.smithi118.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-18T13:38:06.257 INFO:teuthology.orchestra.run.smithi118.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-18T13:38:06.312 INFO:teuthology.orchestra.run.smithi118.stdout:The following NEW packages will be installed: 2024-08-18T13:38:06.312 INFO:teuthology.orchestra.run.smithi118.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-18T13:38:06.313 INFO:teuthology.orchestra.run.smithi118.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-18T13:38:06.313 INFO:teuthology.orchestra.run.smithi118.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-18T13:38:06.313 INFO:teuthology.orchestra.run.smithi118.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-18T13:38:06.314 INFO:teuthology.orchestra.run.smithi118.stdout:The following packages will be upgraded: 2024-08-18T13:38:06.315 INFO:teuthology.orchestra.run.smithi118.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-18T13:38:06.373 INFO:teuthology.orchestra.run.smithi118.stdout:6 upgraded, 23 newly installed, 0 to remove and 289 not upgraded. 2024-08-18T13:38:06.373 INFO:teuthology.orchestra.run.smithi118.stdout:Need to get 10.4 MB of archives. 2024-08-18T13:38:06.373 INFO:teuthology.orchestra.run.smithi118.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-18T13:38:06.373 INFO:teuthology.orchestra.run.smithi118.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-18T13:38:06.505 INFO:teuthology.orchestra.run.smithi118.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-18T13:38:06.507 INFO:teuthology.orchestra.run.smithi118.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-18T13:38:06.508 INFO:teuthology.orchestra.run.smithi118.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-18T13:38:06.513 INFO:teuthology.orchestra.run.smithi118.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-18T13:38:06.514 INFO:teuthology.orchestra.run.smithi118.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-18T13:38:06.516 INFO:teuthology.orchestra.run.smithi178.stdout:Preconfiguring packages ... 2024-08-18T13:38:06.530 INFO:teuthology.orchestra.run.smithi118.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-18T13:38:06.530 INFO:teuthology.orchestra.run.smithi118.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-18T13:38:06.532 INFO:teuthology.orchestra.run.smithi118.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-18T13:38:06.533 INFO:teuthology.orchestra.run.smithi118.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-18T13:38:06.533 INFO:teuthology.orchestra.run.smithi118.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-18T13:38:06.539 INFO:teuthology.orchestra.run.smithi118.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-18T13:38:06.540 INFO:teuthology.orchestra.run.smithi118.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-18T13:38:06.542 INFO:teuthology.orchestra.run.smithi118.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-18T13:38:06.542 INFO:teuthology.orchestra.run.smithi118.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-18T13:38:06.556 INFO:teuthology.orchestra.run.smithi118.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-18T13:38:06.557 INFO:teuthology.orchestra.run.smithi118.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-18T13:38:06.561 INFO:teuthology.orchestra.run.smithi118.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-18T13:38:06.571 INFO:teuthology.orchestra.run.smithi118.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-18T13:38:06.571 INFO:teuthology.orchestra.run.smithi118.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-18T13:38:06.586 INFO:teuthology.orchestra.run.smithi118.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-18T13:38:06.586 INFO:teuthology.orchestra.run.smithi118.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-18T13:38:06.593 INFO:teuthology.orchestra.run.smithi118.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-18T13:38:06.603 INFO:teuthology.orchestra.run.smithi118.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-18T13:38:06.618 INFO:teuthology.orchestra.run.smithi118.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-18T13:38:06.621 INFO:teuthology.orchestra.run.smithi118.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-18T13:38:06.621 INFO:teuthology.orchestra.run.smithi118.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-18T13:38:06.622 INFO:teuthology.orchestra.run.smithi118.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-18T13:38:06.622 INFO:teuthology.orchestra.run.smithi118.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-18T13:38:06.646 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 10.4 MB in 0s (35.4 MB/s) 2024-08-18T13:38:06.918 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-18T13:38:06.922 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-18T13:38:06.947 INFO:teuthology.orchestra.run.smithi118.stdout:Preconfiguring packages ... 2024-08-18T13:38:07.077 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-18T13:38:07.080 INFO:teuthology.orchestra.run.smithi118.stdout:Fetched 10.4 MB in 0s (35.0 MB/s) 2024-08-18T13:38:07.350 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-18T13:38:07.354 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-18T13:38:07.517 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-18T13:38:08.283 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-18T13:38:08.781 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-18T13:38:09.720 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-18T13:38:09.725 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-18T13:38:09.883 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-18T13:38:10.287 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-18T13:38:10.291 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-18T13:38:10.305 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-18T13:38:10.425 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-18T13:38:10.589 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-18T13:38:10.873 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-18T13:38:10.972 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-18T13:38:11.073 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-18T13:38:11.450 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-18T13:38:11.455 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-18T13:38:11.476 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-18T13:38:11.588 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-18T13:38:11.951 INFO:teuthology.orchestra.run.smithi118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-18T13:38:11.956 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-18T13:38:12.077 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-18T13:38:12.149 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-18T13:38:12.210 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-18T13:38:12.618 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-18T13:38:12.630 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-18T13:38:12.763 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-18T13:38:12.764 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-18T13:38:13.294 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-18T13:38:13.312 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-18T13:38:13.342 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-18T13:38:13.352 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-18T13:38:13.468 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-18T13:38:13.688 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package dump. 2024-08-18T13:38:13.707 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-18T13:38:13.746 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking dump (0.4b46-6) ... 2024-08-18T13:38:13.972 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-18T13:38:13.992 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-18T13:38:14.055 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-18T13:38:14.058 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package indent. 2024-08-18T13:38:14.076 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-18T13:38:14.138 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-18T13:38:14.392 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package dump. 2024-08-18T13:38:14.409 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-18T13:38:14.478 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-18T13:38:14.483 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking dump (0.4b46-6) ... 2024-08-18T13:38:14.495 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-18T13:38:14.536 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-18T13:38:14.788 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package indent. 2024-08-18T13:38:14.808 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-18T13:38:14.887 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-18T13:38:14.889 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-18T13:38:14.906 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-18T13:38:14.947 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-18T13:38:15.234 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-18T13:38:15.254 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-18T13:38:15.283 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:15.584 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-18T13:38:15.602 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-18T13:38:15.667 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-18T13:38:16.004 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-18T13:38:16.021 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-18T13:38:16.104 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-18T13:38:16.156 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-18T13:38:16.173 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:16.214 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:16.391 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-18T13:38:16.411 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-18T13:38:16.474 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:16.500 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-18T13:38:16.519 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:16.558 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:16.852 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-18T13:38:16.870 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:16.877 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-18T13:38:16.894 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:16.910 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:16.935 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:17.188 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-18T13:38:17.206 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:17.280 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:17.355 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-18T13:38:17.375 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-18T13:38:17.413 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-18T13:38:17.621 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-18T13:38:17.640 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:17.683 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:17.775 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-18T13:38:17.792 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-18T13:38:17.833 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:18.072 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-18T13:38:18.089 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-18T13:38:18.129 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-18T13:38:18.269 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-18T13:38:18.289 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:18.327 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:18.482 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-18T13:38:18.499 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-18T13:38:18.540 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:18.655 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-18T13:38:18.675 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:18.713 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:18.927 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-18T13:38:18.944 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:18.985 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:18.999 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-18T13:38:19.016 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-18T13:38:19.057 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-18T13:38:19.338 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-18T13:38:19.355 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:19.396 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:19.553 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-18T13:38:19.570 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-18T13:38:19.602 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-18T13:38:19.673 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-18T13:38:19.691 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-18T13:38:19.731 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-18T13:38:19.989 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package modemmanager. 2024-08-18T13:38:20.011 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:20.099 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:20.260 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-18T13:38:20.277 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-18T13:38:20.317 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-18T13:38:20.711 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package wpasupplicant. 2024-08-18T13:38:20.731 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-18T13:38:20.761 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-18T13:38:20.838 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package modemmanager. 2024-08-18T13:38:20.857 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-18T13:38:21.003 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:21.576 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package network-manager. 2024-08-18T13:38:21.593 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-18T13:38:21.683 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:21.804 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package wpasupplicant. 2024-08-18T13:38:21.821 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-18T13:38:21.862 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-18T13:38:22.425 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ppp. 2024-08-18T13:38:22.442 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package network-manager. 2024-08-18T13:38:22.443 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-18T13:38:22.461 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-18T13:38:22.549 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:22.728 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-18T13:38:23.063 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package pptp-linux. 2024-08-18T13:38:23.080 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-18T13:38:23.121 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-18T13:38:23.299 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package ppp. 2024-08-18T13:38:23.320 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-18T13:38:23.541 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-18T13:38:23.562 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-18T13:38:23.583 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-18T13:38:23.626 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-18T13:38:23.919 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-18T13:38:23.928 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package pptp-linux. 2024-08-18T13:38:23.937 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-18T13:38:23.946 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-18T13:38:23.977 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-18T13:38:23.978 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-18T13:38:24.373 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-18T13:38:24.393 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-18T13:38:24.422 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-18T13:38:24.441 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-18T13:38:24.458 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-18T13:38:24.491 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-18T13:38:24.701 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-18T13:38:24.720 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-18T13:38:24.759 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-18T13:38:24.821 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-18T13:38:25.111 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-18T13:38:25.196 INFO:teuthology.orchestra.run.smithi118.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-18T13:38:25.216 INFO:teuthology.orchestra.run.smithi118.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-18T13:38:25.256 INFO:teuthology.orchestra.run.smithi118.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-18T13:38:25.262 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-18T13:38:25.380 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-18T13:38:25.541 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-18T13:38:25.805 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-18T13:38:25.805 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-18T13:38:25.901 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-18T13:38:26.061 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-18T13:38:26.186 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-18T13:38:26.283 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-18T13:38:26.409 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-18T13:38:26.795 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-18T13:38:26.921 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-18T13:38:27.022 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-18T13:38:27.023 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-18T13:38:27.047 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:27.198 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-18T13:38:27.491 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-18T13:38:27.650 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-18T13:38:27.776 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-18T13:38:27.902 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-18T13:38:28.053 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:28.179 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-18T13:38:28.211 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-18T13:38:28.464 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up dump (0.4b46-6) ... 2024-08-18T13:38:28.559 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-18T13:38:28.635 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:28.767 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-18T13:38:28.893 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-18T13:38:29.044 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-18T13:38:29.166 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-18T13:38:29.220 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-18T13:38:29.380 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:29.403 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up dump (0.4b46-6) ... 2024-08-18T13:38:29.497 INFO:teuthology.orchestra.run.smithi118.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-18T13:38:29.506 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:29.578 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:29.665 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-18T13:38:29.713 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-18T13:38:29.833 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:29.839 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-18T13:38:29.981 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-18T13:38:29.984 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:30.149 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-18T13:38:30.298 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:30.426 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:30.552 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-18T13:38:30.725 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:30.725 INFO:teuthology.orchestra.run.smithi178.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-18T13:38:30.725 INFO:teuthology.orchestra.run.smithi178.stdout:which means they are currently configured by ifupdown: 2024-08-18T13:38:30.725 INFO:teuthology.orchestra.run.smithi178.stdout:- enp3s0f1 2024-08-18T13:38:30.725 INFO:teuthology.orchestra.run.smithi178.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-18T13:38:30.725 INFO:teuthology.orchestra.run.smithi178.stdout:remove their configuration from /etc/network/interfaces. 2024-08-18T13:38:30.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:30.754 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:30.880 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-18T13:38:30.889 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-18T13:38:31.214 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-18T13:38:31.610 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:31.610 INFO:teuthology.orchestra.run.smithi118.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-18T13:38:31.610 INFO:teuthology.orchestra.run.smithi118.stdout:which means they are currently configured by ifupdown: 2024-08-18T13:38:31.610 INFO:teuthology.orchestra.run.smithi118.stdout:- enp3s0f1 2024-08-18T13:38:31.610 INFO:teuthology.orchestra.run.smithi118.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-18T13:38:31.610 INFO:teuthology.orchestra.run.smithi118.stdout:remove their configuration from /etc/network/interfaces. 2024-08-18T13:38:31.610 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:31.746 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-18T13:38:31.769 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-18T13:38:32.079 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-18T13:38:32.296 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:32.472 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-18T13:38:32.615 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:32.632 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-18T13:38:32.763 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:32.892 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:33.026 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:33.116 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-18T13:38:33.243 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-18T13:38:33.322 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-18T13:38:33.322 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-18T13:38:33.385 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-18T13:38:33.536 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:33.663 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-18T13:38:33.789 INFO:teuthology.orchestra.run.smithi118.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-18T13:38:33.846 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-18T13:38:33.947 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-18T13:38:34.060 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-18T13:38:34.061 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-18T13:38:34.097 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-18T13:38:34.232 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-18T13:38:34.567 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-18T13:38:34.625 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-18T13:38:34.726 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-18T13:38:34.851 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-18T13:38:35.028 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-18T13:38:35.313 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-18T13:38:38.445 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-18T13:38:38.545 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-18T13:38:39.232 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-18T13:38:39.357 INFO:teuthology.orchestra.run.smithi118.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-18T13:38:39.408 INFO:teuthology.orchestra.run.smithi178.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-18T13:38:39.413 DEBUG:teuthology.parallel:result is None 2024-08-18T13:38:40.204 INFO:teuthology.orchestra.run.smithi118.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-18T13:38:40.208 DEBUG:teuthology.parallel:result is None 2024-08-18T13:38:40.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a17c67c052c1476b20f99d847823755b26703128 2024-08-18T13:38:40.355 DEBUG:teuthology.orchestra.run.smithi118:> dpkg-query -W -f '${Version}' ceph 2024-08-18T13:38:40.379 INFO:teuthology.orchestra.run.smithi118.stdout:17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:38:40.380 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:38:40.380 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-ga17c67c0-1focal is installed. 2024-08-18T13:38:40.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=a17c67c052c1476b20f99d847823755b26703128 2024-08-18T13:38:40.531 DEBUG:teuthology.orchestra.run.smithi178:> dpkg-query -W -f '${Version}' ceph 2024-08-18T13:38:40.555 INFO:teuthology.orchestra.run.smithi178.stdout:17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:38:40.556 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-ga17c67c0-1focal 2024-08-18T13:38:40.556 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-ga17c67c0-1focal is installed. 2024-08-18T13:38:40.558 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-18T13:38:40.558 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:40.558 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-18T13:38:40.575 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:40.576 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-18T13:38:40.592 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-18T13:38:40.592 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:40.592 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2024-08-18T13:38:40.636 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-18T13:38:40.645 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:40.646 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-08-18T13:38:40.662 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-18T13:38:40.717 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-18T13:38:40.718 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:40.718 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-18T13:38:40.733 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-18T13:38:40.744 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:40.744 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-18T13:38:40.759 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-18T13:38:40.770 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-18T13:38:40.770 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:40.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2024-08-18T13:38:40.801 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-18T13:38:40.810 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:40.811 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-08-18T13:38:40.826 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-18T13:38:40.838 INFO:teuthology.run_tasks:Running task ceph... 2024-08-18T13:38:40.921 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-18T13:38:40.921 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /var/log/ceph 2024-08-18T13:38:40.923 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2024-08-18T13:38:40.939 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-18T13:38:40.939 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-18T13:38:40.980 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-18T13:38:40.995 INFO:tasks.ceph:Creating extra log directories... 2024-08-18T13:38:40.996 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-18T13:38:40.998 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-18T13:38:41.058 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-08-18T13:38:41.058 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-08-18T13:38:41.058 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-08-18T13:38:41.058 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:41.058 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-18T13:38:41.065 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-18T13:38:41.148 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:41.148 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-18T13:38:41.155 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-18T13:38:41.199 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-18T13:38:41.199 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-18T13:38:41.200 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093', 'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7859093', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a17c67c052c1476b20f99d847823755b26703128'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a17c67c052c1476b20f99d847823755b26703128'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'sha1': 'a17c67c052c1476b20f99d847823755b26703128'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 5038, 'sha1': 'a17c67c052c1476b20f99d847823755b26703128', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a17c67c052c1476b20f99d847823755b26703128', 'targets': {'smithi118.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi178.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'a17c67c052c1476b20f99d847823755b26703128'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a17c67c052c1476b20f99d847823755b26703128', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-18_13:11:53', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-18T13:38:41.200 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-18T13:38:41.203 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-18T13:38:41.243 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m0777 -- /var/run/ceph 2024-08-18T13:38:41.245 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2024-08-18T13:38:41.260 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:41.261 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2024-08-18T13:38:41.308 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-18T13:38:41.308 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-08-18 13:34:31.034225536 +0000 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-08-18 13:34:30.822230425 +0000 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-08-18 13:34:30.822230425 +0000 2024-08-18T13:38:41.316 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-08-18T13:38:41.317 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-18T13:38:41.325 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-08-18T13:38:41.325 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-08-18T13:38:41.325 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000301361 s, 1.7 MB/s 2024-08-18T13:38:41.326 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-18T13:38:41.378 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2024-08-18T13:38:41.385 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-18T13:38:41.385 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.385 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-08-18T13:38:41.386 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.386 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-08-18 13:34:31.462215666 +0000 2024-08-18T13:38:41.386 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-08-18 13:34:31.250220554 +0000 2024-08-18T13:38:41.386 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-08-18 13:34:31.250220554 +0000 2024-08-18T13:38:41.386 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-08-18T13:38:41.386 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-18T13:38:41.396 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-08-18T13:38:41.396 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-08-18T13:38:41.396 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.00029725 s, 1.7 MB/s 2024-08-18T13:38:41.397 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-18T13:38:41.450 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2024-08-18T13:38:41.458 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-18T13:38:41.458 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.458 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-08-18T13:38:41.458 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.459 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-08-18 13:34:31.878206071 +0000 2024-08-18T13:38:41.459 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-08-18 13:34:31.670210868 +0000 2024-08-18T13:38:41.459 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-08-18 13:34:31.670210868 +0000 2024-08-18T13:38:41.459 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-08-18T13:38:41.459 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-18T13:38:41.474 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-08-18T13:38:41.474 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-08-18T13:38:41.474 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000270277 s, 1.9 MB/s 2024-08-18T13:38:41.475 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-18T13:38:41.484 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2024-08-18T13:38:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-18T13:38:41.532 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.533 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-08-18T13:38:41.533 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.533 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-08-18 13:34:32.318195925 +0000 2024-08-18T13:38:41.533 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-08-18 13:34:32.094201091 +0000 2024-08-18T13:38:41.533 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-08-18 13:34:32.094201091 +0000 2024-08-18T13:38:41.533 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-08-18T13:38:41.533 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-18T13:38:41.545 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-08-18T13:38:41.545 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-08-18T13:38:41.545 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000295267 s, 1.7 MB/s 2024-08-18T13:38:41.546 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-18T13:38:41.556 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:41.556 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:41.556 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-08-18T13:38:41.563 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-18T13:38:41.563 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-08-18 13:34:31.042029131 +0000 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-08-18 13:34:30.838033833 +0000 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-08-18 13:34:30.838033833 +0000 2024-08-18T13:38:41.611 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-08-18T13:38:41.612 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-18T13:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-08-18T13:38:41.623 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-08-18T13:38:41.624 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000298424 s, 1.7 MB/s 2024-08-18T13:38:41.625 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-18T13:38:41.633 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-08-18T13:38:41.639 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-18T13:38:41.639 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.639 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-08-18T13:38:41.639 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.639 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-08-18 13:34:31.438020004 +0000 2024-08-18T13:38:41.640 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-08-18 13:34:31.238024614 +0000 2024-08-18T13:38:41.640 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-08-18 13:34:31.238024614 +0000 2024-08-18T13:38:41.640 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-08-18T13:38:41.640 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-18T13:38:41.694 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-08-18T13:38:41.694 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-08-18T13:38:41.695 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000293137 s, 1.7 MB/s 2024-08-18T13:38:41.696 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-18T13:38:41.706 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-08-18 13:34:31.850010508 +0000 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-08-18 13:34:31.654015025 +0000 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-08-18 13:34:31.654015025 +0000 2024-08-18T13:38:41.712 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-08-18T13:38:41.712 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-18T13:38:41.770 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-08-18T13:38:41.770 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-08-18T13:38:41.770 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.00030356 s, 1.7 MB/s 2024-08-18T13:38:41.771 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-18T13:38:41.778 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-08-18 13:34:32.286000457 +0000 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-08-18 13:34:32.058005713 +0000 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-08-18 13:34:32.058005713 +0000 2024-08-18T13:38:41.784 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-08-18T13:38:41.784 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-18T13:38:41.843 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-08-18T13:38:41.843 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-08-18T13:38:41.843 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.0003045 s, 1.7 MB/s 2024-08-18T13:38:41.844 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-18T13:38:41.852 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:41.852 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-18T13:38:41.852 INFO:tasks.ceph:Generating config... 2024-08-18T13:38:41.854 INFO:tasks.ceph:[client] client mount timeout = 600 2024-08-18T13:38:41.854 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-08-18T13:38:41.854 INFO:tasks.ceph:[client] debug client = 20 2024-08-18T13:38:41.854 INFO:tasks.ceph:[client] debug ms = 1 2024-08-18T13:38:41.854 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-08-18T13:38:41.854 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-08-18T13:38:41.854 INFO:tasks.ceph:[mds] debug mds = 20 2024-08-18T13:38:41.854 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-08-18T13:38:41.854 INFO:tasks.ceph:[mds] debug ms = 1 2024-08-18T13:38:41.854 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-08-18T13:38:41.854 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds debug frag = True 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = False 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mgr] debug client = 20 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-18T13:38:41.855 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-18T13:38:41.856 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-18T13:38:41.856 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-18T13:38:41.856 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-18T13:38:41.856 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-18T13:38:41.856 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-18T13:38:41.857 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-08-18T13:38:41.857 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-18T13:38:41.857 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-08-18T13:38:41.857 INFO:tasks.ceph:Setting up mon.a... 2024-08-18T13:38:41.857 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-18T13:38:41.893 INFO:teuthology.orchestra.run.smithi118.stdout:creating /etc/ceph/ceph.keyring 2024-08-18T13:38:41.895 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-18T13:38:41.927 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-18T13:38:41.943 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.118'), ('mon.b', '172.21.15.178'), ('mon.c', '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]')] 2024-08-18T13:38:41.944 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.118,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-08-18T13:38:41.944 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:41.945 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-18T13:38:41.951 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.118 --add b 172.21.15.178 --addv c '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool: generated fsid ac1e0c0e-66a6-4fa9-9612-9b35e09366c7 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:setting min_mon_release = octopus 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:epoch 0 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:fsid ac1e0c0e-66a6-4fa9-9612-9b35e09366c7 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:last_changed 2024-08-18T13:38:41.987269+0000 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:created 2024-08-18T13:38:41.987269+0000 2024-08-18T13:38:41.988 INFO:teuthology.orchestra.run.smithi118.stdout:min_mon_release 15 (octopus) 2024-08-18T13:38:41.989 INFO:teuthology.orchestra.run.smithi118.stdout:election_strategy: 1 2024-08-18T13:38:41.989 INFO:teuthology.orchestra.run.smithi118.stdout:0: [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon.a 2024-08-18T13:38:41.989 INFO:teuthology.orchestra.run.smithi118.stdout:1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2024-08-18T13:38:41.989 INFO:teuthology.orchestra.run.smithi118.stdout:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-08-18T13:38:41.989 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-08-18T13:38:41.990 DEBUG:teuthology.orchestra.run.smithi118:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-18T13:38:42.039 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ac1e0c0e-66a6-4fa9-9612-9b35e09366c7... 2024-08-18T13:38:42.041 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-18T13:38:42.043 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-18T13:38:42.076 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-08-18T13:38:42.076 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-08-18T13:38:42.076 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-18T13:38:42.076 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-08-18T13:38:42.076 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.076 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.077 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-08-18T13:38:42.078 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-08-18T13:38:42.079 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.118,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = ac1e0c0e-66a6-4fa9-9612-9b35e09366c7 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-08-18T13:38:42.080 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-08-18T13:38:42.081 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 20 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 20 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 10 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-08-18T13:38:42.082 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: osd max markdown count = 1000 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-08-18T13:38:42.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-18T13:38:42.084 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-08-18T13:38:42.085 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: client_shutdown_timeout = 120 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-08-18T13:38:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.e] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.f] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout: mds bal fragment size max = 10000 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout: mds bal merge size = 5 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout: mds bal split bits = 3 2024-08-18T13:38:42.087 INFO:teuthology.orchestra.run.smithi178.stdout: mds bal split size = 100 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = True 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: mds oft prefetch dirfrags = False 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = True 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: mds_sleep_rank_change = 5000000.0 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-08-18T13:38:42.088 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: chdir = "" 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: auth supported = cephx 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: filestore xattr use omap = true 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: mon clock drift allowed = 1.000 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: osd crush chooseleaf type = 0 2024-08-18T13:38:42.090 INFO:teuthology.orchestra.run.smithi118.stdout: auth debug = true 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on old message = true 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: ms die on bug = true 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: mon pg warn max object skew = 0 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: # disable pg_autoscaler by default for new pools 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default size = 2 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow primary affinity = true 2024-08-18T13:38:42.091 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd allow pg remap = true 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on legacy crush tunables = false 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on crush straw calc version zero = false 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on no sortbitwise = false 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on osd down out interval zero = false 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon warn on too few osds = false 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon_allow_pool_size_one = true 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: osd default data pool replay window = 5 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.092 INFO:teuthology.orchestra.run.smithi118.stdout: mon allow pool delete = true 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: mon cluster log file level = debug 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: debug asserts on shutdown = true 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: mon health detail to clog = false 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: mon host = "172.21.15.118,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = ac1e0c0e-66a6-4fa9-9612-9b35e09366c7 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout:[osd] 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: osd journal size = 100 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub load threshold = 5.0 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: osd scrub max interval = 600 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: osd mclock profile = high_recovery_ops 2024-08-18T13:38:42.093 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd recover clone overlap = true 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd recovery max chunk = 1048576 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug shutdown = true 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug op order = true 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug verify stray on activate = true 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd open classes on start = true 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug pg log writeout = true 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd deep scrub update digest min age = 30 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.094 INFO:teuthology.orchestra.run.smithi118.stdout: osd map max advance = 10 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: journal zero on create = true 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: filestore ondisk finisher threads = 3 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: filestore apply finisher threads = 3 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: bdev debug aio = true 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: osd debug misdirected ops = true 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore block size = 96636764160 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: bluestore fsck on mount = True 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluefs = 20 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: debug bluestore = 20 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: debug osd = 20 2024-08-18T13:38:42.095 INFO:teuthology.orchestra.run.smithi118.stdout: debug rocksdb = 10 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd backfillfull_ratio = 0.85 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd full ratio = 0.9 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd nearfull ratio = 0.8 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: osd failsafe full ratio = 0.95 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: osd max markdown count = 1000 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: osd objectstore = bluestore 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr] 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: debug mgr = 20 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2024-08-18T13:38:42.096 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: mgr/telemetry/nag = false 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: debug client = 20 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout:[mon] 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: debug mon = 20 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: debug paxos = 20 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: debug auth = 20 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: mon data avail warn = 5 2024-08-18T13:38:42.097 INFO:teuthology.orchestra.run.smithi118.stdout: mon mgr mkfs grace = 240 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min pgs per osd = 4 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd reporter subtree level = osd 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: mon osd prime pg temp = true 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: mon reweight min bytes per osd = 10 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: auth mon ticket ttl = 660 # 11m 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: auth service ticket ttl = 240 # 4m 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.098 INFO:teuthology.orchestra.run.smithi118.stdout: # don't complain about insecure global_id in the test suite 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: # 1m isn't quite enough 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: mon_down_mkfs_grace = 2m 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: mon_warn_on_filestore_osds = false 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: mon op complaint time = 120 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout:[client] 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: rgw cache enabled = true 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable ops log = true 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: rgw enable usage log = true 2024-08-18T13:38:42.099 INFO:teuthology.orchestra.run.smithi118.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout: client mount timeout = 600 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout: client_shutdown_timeout = 120 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout: debug client = 20 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 900 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 900 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.a] 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.b] 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:[mon.c] 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.e] 2024-08-18T13:38:42.100 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.d] 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.f] 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout:[mds] 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: debug mds = 20 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: debug mds balancer = 20 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: debug ms = 1 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal fragment size max = 10000 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal merge size = 5 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split bits = 3 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds bal split size = 100 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug frag = True 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds debug scatterstat = True 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds oft prefetch dirfrags = False 2024-08-18T13:38:42.101 INFO:teuthology.orchestra.run.smithi118.stdout: mds op complaint time = 180 2024-08-18T13:38:42.102 INFO:teuthology.orchestra.run.smithi118.stdout: mds verify scatter = True 2024-08-18T13:38:42.102 INFO:teuthology.orchestra.run.smithi118.stdout: mds_sleep_rank_change = 5000000.0 2024-08-18T13:38:42.102 INFO:teuthology.orchestra.run.smithi118.stdout: osd op complaint time = 180 2024-08-18T13:38:42.102 INFO:teuthology.orchestra.run.smithi118.stdout: rados mon op timeout = 900 2024-08-18T13:38:42.102 INFO:teuthology.orchestra.run.smithi118.stdout: rados osd op timeout = 900 2024-08-18T13:38:42.102 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-18T13:38:42.102 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-18T13:38:42.184 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-18T13:38:42.184 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:42.184 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-18T13:38:42.190 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:42.191 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-18T13:38:42.197 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:38:42.197 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:42.197 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-18T13:38:42.197 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-18T13:38:42.220 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:38:42.220 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:42.268 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:38:42.268 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:42.268 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-18T13:38:42.268 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-18T13:38:42.296 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:38:42.296 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:42.301 INFO:tasks.ceph:Setting up mon nodes... 2024-08-18T13:38:42.302 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-18T13:38:42.302 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-08-18T13:38:42.347 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-08-18T13:38:42.349 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-18T13:38:42.396 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-18T13:38:42.399 INFO:tasks.ceph:Setting up mds nodes... 2024-08-18T13:38:42.399 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-08-18T13:38:42.446 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-08-18T13:38:42.448 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-08-18T13:38:42.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-08-18T13:38:42.498 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-08-18T13:38:42.502 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-08-18T13:38:42.512 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-08-18T13:38:42.597 INFO:teuthology.orchestra.run.smithi118.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-08-18T13:38:42.600 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-08-18T13:38:42.613 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-08-18T13:38:42.659 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-08-18T13:38:42.661 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-08-18T13:38:42.672 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-08-18T13:38:42.711 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-08-18T13:38:42.714 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-08-18T13:38:42.773 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-08-18T13:38:42.812 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-08-18T13:38:42.815 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-08-18T13:38:42.828 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-18T13:38:42.828 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-18T13:38:42.863 INFO:teuthology.orchestra.run.smithi118.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-18T13:38:42.876 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-18T13:38:42.876 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi118.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-18T13:38:42.877 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-18T13:38:42.934 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:42.934 INFO:tasks.ceph:role: osd.0 2024-08-18T13:38:42.934 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:38:42.934 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-18T13:38:42.976 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:42.977 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:42.979 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-08-18T13:38:42.979 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-18T13:38:43.001 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-18T13:38:43.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:43.009 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:43.009 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-18T13:38:43.067 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:43.067 INFO:tasks.ceph:role: osd.1 2024-08-18T13:38:43.067 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:38:43.068 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-18T13:38:43.111 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:43.111 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:43.111 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:43.111 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-08-18T13:38:43.111 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:43.112 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:43.112 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:43.112 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:43.112 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:43.112 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:43.113 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-08-18T13:38:43.113 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-18T13:38:43.130 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-18T13:38:43.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:43.190 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:43.191 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-18T13:38:43.201 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:43.201 INFO:tasks.ceph:role: osd.2 2024-08-18T13:38:43.201 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:38:43.201 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-18T13:38:43.241 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:43.242 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:43.243 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-08-18T13:38:43.243 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-18T13:38:43.307 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-18T13:38:43.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:43.314 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:43.314 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-18T13:38:43.325 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:43.325 INFO:tasks.ceph:role: osd.3 2024-08-18T13:38:43.325 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:38:43.325 DEBUG:teuthology.orchestra.run.smithi118:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-18T13:38:43.365 INFO:teuthology.orchestra.run.smithi118.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout: = reflink=1 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:43.366 INFO:teuthology.orchestra.run.smithi118.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:43.368 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi118.front.sepia.ceph.com -o noatime 2024-08-18T13:38:43.368 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-08-18T13:38:43.387 DEBUG:teuthology.orchestra.run.smithi118:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-18T13:38:43.441 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:43.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:43.442 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:43.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:43.511+0000 7fe4ee3bc3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-18T13:38:43.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:43.511+0000 7fe4ee3bc3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-18T13:38:43.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:43.511+0000 7fe4ee3bc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-18T13:38:43.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:43.511+0000 7fe4ee3bc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-18T13:38:43.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:43.511+0000 7fe4ee3bc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-18T13:38:43.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:43.511+0000 7fe4ee3bc3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-18T13:38:44.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:44.071+0000 7fe4ee3bc3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:44.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:44.863+0000 7fe4ee3bc3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:45.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:45.651+0000 7fe4ee3bc3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:46.513 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-18T13:38:46.529 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:46.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:46.559+0000 7f0268eb63c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-18T13:38:46.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:46.559+0000 7f0268eb63c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-18T13:38:46.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:46.559+0000 7f0268eb63c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-18T13:38:46.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:46.559+0000 7f0268eb63c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-18T13:38:46.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:46.559+0000 7f0268eb63c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-18T13:38:46.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:46.563+0000 7f0268eb63c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-18T13:38:47.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:47.119+0000 7f0268eb63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:47.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:47.907+0000 7f0268eb63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:48.707 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:48.703+0000 7f0268eb63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:49.557 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-18T13:38:49.618 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:49.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:49.651+0000 7f8dacc033c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-18T13:38:49.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:49.651+0000 7f8dacc033c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-18T13:38:49.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:49.651+0000 7f8dacc033c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-18T13:38:49.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:49.651+0000 7f8dacc033c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-18T13:38:49.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:49.651+0000 7f8dacc033c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-18T13:38:49.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:49.655+0000 7f8dacc033c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-18T13:38:50.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:50.207+0000 7f8dacc033c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:50.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:50.987+0000 7f8dacc033c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:51.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:51.767+0000 7f8dacc033c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:52.625 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-18T13:38:52.641 DEBUG:teuthology.orchestra.run.smithi118:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:52.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:52.671+0000 7f3c32dc13c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-18T13:38:52.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:52.671+0000 7f3c32dc13c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-18T13:38:52.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:52.671+0000 7f3c32dc13c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-18T13:38:52.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:52.671+0000 7f3c32dc13c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-18T13:38:52.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:52.671+0000 7f3c32dc13c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-18T13:38:52.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:52.675+0000 7f3c32dc13c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-18T13:38:53.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:53.231+0000 7f3c32dc13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:54.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:54.019+0000 7f3c32dc13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:54.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:38:54.803+0000 7f3c32dc13c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:55.649 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-18T13:38:55.706 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-18T13:38:55.723 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:55.723 INFO:tasks.ceph:role: osd.4 2024-08-18T13:38:55.724 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:38:55.724 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-18T13:38:55.807 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:55.808 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:55.810 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-08-18T13:38:55.810 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-08-18T13:38:55.828 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-18T13:38:55.840 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:55.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:55.841 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-18T13:38:55.851 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:55.851 INFO:tasks.ceph:role: osd.5 2024-08-18T13:38:55.851 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:38:55.852 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-18T13:38:55.939 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:55.939 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:55.939 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:55.939 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 2024-08-18T13:38:55.939 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:55.940 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:55.940 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:55.940 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:55.940 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:55.940 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:55.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-08-18T13:38:55.941 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-08-18T13:38:55.957 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-18T13:38:55.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:55.967 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:55.967 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-08-18T13:38:55.977 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:55.977 INFO:tasks.ceph:role: osd.6 2024-08-18T13:38:55.977 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:38:55.977 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-18T13:38:56.017 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:56.017 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:56.017 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:56.017 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 2024-08-18T13:38:56.017 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:56.017 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:56.018 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:56.018 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:56.018 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:56.018 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:56.019 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-08-18T13:38:56.019 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-08-18T13:38:56.086 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-08-18T13:38:56.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:56.093 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:56.093 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-08-18T13:38:56.104 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-18T13:38:56.104 INFO:tasks.ceph:role: osd.7 2024-08-18T13:38:56.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:38:56.104 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-18T13:38:56.191 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-18T13:38:56.191 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-18T13:38:56.191 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-18T13:38:56.191 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 2024-08-18T13:38:56.192 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-18T13:38:56.192 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-08-18T13:38:56.192 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-18T13:38:56.192 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-18T13:38:56.192 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-18T13:38:56.192 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-18T13:38:56.193 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-08-18T13:38:56.193 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-08-18T13:38:56.208 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-08-18T13:38:56.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:38:56.218 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-08-18T13:38:56.219 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:56.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:56.336+0000 7f68eab143c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-18T13:38:56.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:56.336+0000 7f68eab143c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-18T13:38:56.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:56.336+0000 7f68eab143c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-18T13:38:56.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:56.336+0000 7f68eab143c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-18T13:38:56.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:56.336+0000 7f68eab143c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-18T13:38:56.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:56.336+0000 7f68eab143c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-18T13:38:56.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:56.888+0000 7f68eab143c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:57.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:57.680+0000 7f68eab143c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:58.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:58.464+0000 7f68eab143c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:38:59.328 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-18T13:38:59.343 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:38:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:59.376+0000 7f226ea433c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-18T13:38:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:59.376+0000 7f226ea433c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-18T13:38:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:59.376+0000 7f226ea433c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-18T13:38:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:59.376+0000 7f226ea433c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-18T13:38:59.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:59.376+0000 7f226ea433c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-18T13:38:59.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:59.376+0000 7f226ea433c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-18T13:38:59.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:38:59.928+0000 7f226ea433c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:00.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:00.712+0000 7f226ea433c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:01.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:01.500+0000 7f226ea433c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:02.388 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-18T13:39:02.445 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:39:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:02.472+0000 7f16cca133c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-08-18T13:39:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:02.472+0000 7f16cca133c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-08-18T13:39:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:02.472+0000 7f16cca133c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-18T13:39:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:02.472+0000 7f16cca133c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-18T13:39:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:02.472+0000 7f16cca133c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-18T13:39:02.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:02.472+0000 7f16cca133c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-08-18T13:39:03.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:03.032+0000 7f16cca133c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:03.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:03.816+0000 7f16cca133c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:04.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:04.600+0000 7f16cca133c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:05.455 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-08-18T13:39:05.472 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:39:05.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:05.500+0000 7f0537b3f3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-08-18T13:39:05.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:05.500+0000 7f0537b3f3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-08-18T13:39:05.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:05.500+0000 7f0537b3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-18T13:39:05.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:05.500+0000 7f0537b3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-18T13:39:05.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:05.500+0000 7f0537b3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-18T13:39:05.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:05.504+0000 7f0537b3f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-08-18T13:39:06.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:06.056+0000 7f0537b3f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:06.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:06.840+0000 7f0537b3f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:07.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:07.632+0000 7f0537b3f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-18T13:39:08.499 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-08-18T13:39:08.556 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-18T13:39:08.556 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.556 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-08-18T13:39:08.573 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.573 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-08-18T13:39:08.584 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.584 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-08-18T13:39:08.641 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.641 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-08-18T13:39:08.655 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.655 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-18T13:39:08.666 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.666 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-18T13:39:08.725 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.725 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-18T13:39:08.736 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.736 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-18T13:39:08.749 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.749 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-18T13:39:08.766 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.766 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-08-18T13:39:08.824 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.824 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-08-18T13:39:08.834 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.834 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-08-18T13:39:08.849 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.849 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-18T13:39:08.905 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.906 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-18T13:39:08.916 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.916 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-08-18T13:39:08.928 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:08.928 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-08-18T13:39:08.988 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:08.988 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-18T13:39:08.994 INFO:tasks.ceph:Adding keys to all mons... 2024-08-18T13:39:08.995 DEBUG:teuthology.orchestra.run.smithi118:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-18T13:39:08.997 DEBUG:teuthology.orchestra.run.smithi178:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.y] 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFmR8CkFBAAywV73cT3zGlCeNLr6RoAUw== 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.a] 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFm63SQGhAA5blsCxtzxaQHgyivdcfH2w== 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.c] 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFmbJ6xHRAAr3a2PskDtHCPL0ylxaPymQ== 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.e] 2024-08-18T13:39:09.009 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFmChGRIxAA+5x9eTQc+Mf+BHDR+9PfSQ== 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.0] 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBj+cFmoBCgHhAAMWRjbv9tVzA0yw30ctznrg== 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.1] 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBm+cFmv7udIRAAdqFtvDzxI1rL0+yLZSUl9Q== 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.2] 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBp+cFmkTwaJxAAAPUZVt8wER+k/uAQeqD5og== 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.3] 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBs+cFmx6ZSKBAA8aE+I63/Nibfco7PZTyp2g== 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout:[mgr.x] 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFm1XedFxAAaj1DE1JDA52fSEDO/p3Jpg== 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.b] 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFmygdCJxAAgQb66KFxud8OaRlj7qNXwQ== 2024-08-18T13:39:09.010 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.d] 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFmoqJaKhAADp1MIT8PQD2XtGfDmto+Xw== 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout:[mds.f] 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFmgSliMBAA9dH/5F3aol8230+ehoGylg== 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.4] 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBw+cFmclVWFBAA0QwyTWvzWl+ySkrkMHeEoQ== 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.5] 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBz+cFmFVK3FhAA37CD6IaviDmcHwHLEB8Sew== 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.6] 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQB2+cFm2fN6HBAAUYBnK6x/9h6yGLf+jwu2DA== 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout:[osd.7] 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQB5+cFmxZ4+HhAAbDIptWJsL9p26eQ7g94DoQ== 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout:[client.0] 2024-08-18T13:39:09.011 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQBi+cFmT2RvMxAAYqut95I8FFYTrVQei5daxg== 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.y] 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFmR8CkFBAAywV73cT3zGlCeNLr6RoAUw== 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFm63SQGhAA5blsCxtzxaQHgyivdcfH2w== 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFmbJ6xHRAAr3a2PskDtHCPL0ylxaPymQ== 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.e] 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFmChGRIxAA+5x9eTQc+Mf+BHDR+9PfSQ== 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.0] 2024-08-18T13:39:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBj+cFmoBCgHhAAMWRjbv9tVzA0yw30ctznrg== 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.1] 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBm+cFmv7udIRAAdqFtvDzxI1rL0+yLZSUl9Q== 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.2] 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBp+cFmkTwaJxAAAPUZVt8wER+k/uAQeqD5og== 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.3] 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBs+cFmx6ZSKBAA8aE+I63/Nibfco7PZTyp2g== 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.x] 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFm1XedFxAAaj1DE1JDA52fSEDO/p3Jpg== 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFmygdCJxAAgQb66KFxud8OaRlj7qNXwQ== 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-08-18T13:39:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFmoqJaKhAADp1MIT8PQD2XtGfDmto+Xw== 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.f] 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFmgSliMBAA9dH/5F3aol8230+ehoGylg== 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.4] 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBw+cFmclVWFBAA0QwyTWvzWl+ySkrkMHeEoQ== 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.5] 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBz+cFmFVK3FhAA37CD6IaviDmcHwHLEB8Sew== 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.6] 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB2+cFm2fN6HBAAUYBnK6x/9h6yGLf+jwu2DA== 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.7] 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB5+cFmxZ4+HhAAbDIptWJsL9p26eQ7g94DoQ== 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2024-08-18T13:39:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQBi+cFmT2RvMxAAYqut95I8FFYTrVQei5daxg== 2024-08-18T13:39:09.016 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-18T13:39:09.018 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-18T13:39:09.060 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.103 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.151 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.153 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.201 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.240 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.289 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.291 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.333 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.335 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.379 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.419 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.464 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.467 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.552 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-18T13:39:09.555 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-18T13:39:09.598 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.643 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.693 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.695 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.739 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.740 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-18T13:39:09.787 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.827 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.871 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.874 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.956 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:09.958 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:10.002 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:10.005 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-18T13:39:10.047 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-18T13:39:10.091 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-18T13:39:10.140 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-18T13:39:10.140 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-18T13:39:10.154 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-18T13:39:10.207 DEBUG:teuthology.orchestra.run.smithi118:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-18T13:39:10.219 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-18T13:39:10.236 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-18T13:39:10.285 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-18T13:39:10.299 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-18T13:39:10.312 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-18T13:39:10.365 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-18T13:39:10.378 DEBUG:teuthology.orchestra.run.smithi118:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:39:10.380 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-18T13:39:10.385 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-18T13:39:10.385 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-18T13:39:10.385 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-18T13:39:10.387 INFO:tasks.ceph.mon.a:Started 2024-08-18T13:39:10.388 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-18T13:39:10.388 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-18T13:39:10.429 INFO:tasks.ceph.mon.b:Started 2024-08-18T13:39:10.430 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-18T13:39:10.430 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-18T13:39:10.432 INFO:tasks.ceph.mon.c:Started 2024-08-18T13:39:10.432 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-18T13:39:10.432 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-08-18T13:39:10.432 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-08-18T13:39:10.435 INFO:tasks.ceph.mgr.y:Started 2024-08-18T13:39:10.435 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-18T13:39:10.435 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-18T13:39:10.437 INFO:tasks.ceph.mgr.x:Started 2024-08-18T13:39:10.438 INFO:tasks.ceph:Setting crush tunables to default 2024-08-18T13:39:10.438 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph osd crush tunables default 2024-08-18T13:39:10.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf21cc9700 1 Processor -- start 2024-08-18T13:39:10.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf21cc9700 1 -- start start 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf21cc9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf21cc9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0c3de0 con 0x7fbf1c14be20 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf21cc9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c06d620 con 0x7fbf1c0c2f80 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf21cc9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c0bfb90 con 0x7fbf1c0c3350 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf214c8700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbf1c14be20 0x7fbf1c15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:33858/0 (socket says 172.21.15.118:33858) 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf214c8700 1 -- 172.21.15.118:0/2509416178 learned_addr learned my addr 172.21.15.118:0/2509416178 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.535+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.539+0000 7fbf1bfff700 1 -- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 msgr2=0x7fbf1c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.539+0000 7fbf1bfff700 1 -- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 msgr2=0x7fbf1c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.539+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-18T13:39:10.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.539+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T13:39:10.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.743+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.743+0000 7fbf1bfff700 1 -- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 msgr2=0x7fbf1c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-18T13:39:10.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.743+0000 7fbf1bfff700 1 -- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 msgr2=0x7fbf1c14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-18T13:39:10.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.743+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-18T13:39:10.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.743+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355395097 0 0) 0x7fbf1c0c3de0 con 0x7fbf1c14be20 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf04003570 con 0x7fbf1c14be20 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944828270 0 0) 0x7fbf1c06d620 con 0x7fbf1c0c2f80 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf1c0c3de0 con 0x7fbf1c0c2f80 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2567397849 0 0) 0x7fbf1c0c3de0 con 0x7fbf1c0c2f80 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 msgr2=0x7fbf1c14b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 --2- 172.21.15.118:0/2509416178 >> v2:172.21.15.178:3300/0 conn(0x7fbf1c0c3350 0x7fbf1c14b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 >> v1:172.21.15.118:6789/0 conn(0x7fbf1c14be20 legacy=0x7fbf1c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:10.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0c4980 con 0x7fbf1c0c2f80 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fbf140017f0 con 0x7fbf1c0c2f80 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 >> v1:172.21.15.178:6789/0 conn(0x7fbf1c0c2f80 legacy=0x7fbf1c0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf04005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf040037b0 0x7fbf04003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf040041d0 0x7fbf0400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0bfb90 con 0x7fbf04004c50 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0400c200 con 0x7fbf040037b0 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0400c3c0 con 0x7fbf040041d0 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf20cc7700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf04005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf040037b0 0x7fbf04003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf214c8700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf040041d0 0x7fbf0400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf20cc7700 1 -- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf040041d0 msgr2=0x7fbf0400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:10.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf20cc7700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf040041d0 0x7fbf0400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf20cc7700 1 -- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf040037b0 msgr2=0x7fbf04003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:10.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf20cc7700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf040037b0 0x7fbf04003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf214c8700 1 -- 172.21.15.118:0/2509416178 reap_dead start 2024-08-18T13:39:10.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf20cc7700 1 -- 172.21.15.118:0/2509416178 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0c3de0 con 0x7fbf04004c50 2024-08-18T13:39:10.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.751+0000 7fbf20cc7700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf04005040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d5c0 tx=0x7fbf1c0c4bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bb26b576ebcfc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:10.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.755+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14010460 con 0x7fbf04004c50 2024-08-18T13:39:10.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.755+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140105e0 con 0x7fbf04004c50 2024-08-18T13:39:10.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.755+0000 7fbf1b7fe700 1 -- 172.21.15.118:0/2509416178 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140109a0 con 0x7fbf04004c50 2024-08-18T13:39:10.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2509416178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 msgr2=0x7fbf04005040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:10.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf04005040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d5c0 tx=0x7fbf1c0c4bd0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.763 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2509416178 shutdown_connections 2024-08-18T13:39:10.763 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 --2- 172.21.15.118:0/2509416178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf04005040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.763 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2509416178 >> 172.21.15.118:0/2509416178 conn(0x7fbf1c06c750 msgr2=0x7fbf1c0c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:10.763 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2509416178 shutdown_connections 2024-08-18T13:39:10.763 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2509416178 wait complete. 2024-08-18T13:39:10.763 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 Processor -- start 2024-08-18T13:39:10.763 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- start start 2024-08-18T13:39:10.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf1c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:10.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1508d0 0x7fbf1c0683c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:10.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c147b20 0x7fbf1c163e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:10.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1468f0 con 0x7fbf04004c50 2024-08-18T13:39:10.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c146dc0 con 0x7fbf1c1508d0 2024-08-18T13:39:10.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf21cc9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c150440 con 0x7fbf1c147b20 2024-08-18T13:39:10.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf1c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf1c0c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36378/0 (socket says 172.21.15.118:36378) 2024-08-18T13:39:10.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 -- 172.21.15.118:0/2406565399 learned_addr learned my addr 172.21.15.118:0/2406565399 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:10.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf214c8700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c147b20 0x7fbf1c163e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf1bfff700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1508d0 0x7fbf1c0683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:10.767 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 -- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c147b20 msgr2=0x7fbf1c163e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:10.767 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c147b20 0x7fbf1c163e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.767 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 -- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1508d0 msgr2=0x7fbf1c0683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:10.767 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1508d0 0x7fbf1c0683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.767 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.759+0000 7fbf20cc7700 1 -- 172.21.15.118:0/2406565399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c146b20 con 0x7fbf04004c50 2024-08-18T13:39:10.767 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf20cc7700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf1c0c6750 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400a220 tx=0x7fbf14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:10.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf18ff9700 1 -- 172.21.15.118:0/2406565399 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14020070 con 0x7fbf04004c50 2024-08-18T13:39:10.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c14fb10 con 0x7fbf04004c50 2024-08-18T13:39:10.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf18ff9700 1 -- 172.21.15.118:0/2406565399 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf14010460 con 0x7fbf04004c50 2024-08-18T13:39:10.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c1500d0 con 0x7fbf04004c50 2024-08-18T13:39:10.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf18ff9700 1 -- 172.21.15.118:0/2406565399 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14019890 con 0x7fbf04004c50 2024-08-18T13:39:10.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf18ff9700 1 -- 172.21.15.118:0/2406565399 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fbf14019c00 con 0x7fbf04004c50 2024-08-18T13:39:10.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.763+0000 7fbf18ff9700 1 -- 172.21.15.118:0/2406565399 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbf14025bb0 con 0x7fbf04004c50 2024-08-18T13:39:10.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.767+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c071410 con 0x7fbf04004c50 2024-08-18T13:39:10.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.771+0000 7fbf18ff9700 1 -- 172.21.15.118:0/2406565399 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fbf140447a0 con 0x7fbf04004c50 2024-08-18T13:39:10.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.839+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbf1c1468f0 con 0x7fbf04004c50 2024-08-18T13:39:10.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.843+0000 7fbf18ff9700 1 -- 172.21.15.118:0/2406565399 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbf14033070 con 0x7fbf04004c50 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 msgr2=0x7fbf1c0c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf1c0c6750 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400a220 tx=0x7fbf14017000 comp rx=0 tx=0).stop 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 shutdown_connections 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1c147b20 0x7fbf1c163e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf1c1508d0 0x7fbf1c0683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 --2- 172.21.15.118:0/2406565399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf04004c50 0x7fbf1c0c6750 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 >> 172.21.15.118:0/2406565399 conn(0x7fbf1c06c750 msgr2=0x7fbf1c156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 shutdown_connections 2024-08-18T13:39:10.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:10.847+0000 7fbf21cc9700 1 -- 172.21.15.118:0/2406565399 wait complete. 2024-08-18T13:39:10.852 INFO:teuthology.orchestra.run.smithi118.stderr:adjusted tunables profile to default 2024-08-18T13:39:10.862 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-18T13:39:10.863 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:10.863 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-18T13:39:10.871 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:10.871 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-18T13:39:10.884 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:10.885 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-18T13:39:10.896 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:39:10.896 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-18T13:39:10.936 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:10.931+0000 7f3a305dc000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-18T13:39:10.949 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:10.949 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-18T13:39:10.962 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:10.956+0000 7fdfd8c41000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-18T13:39:10.966 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:10.966 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-18T13:39:11.021 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:11.021 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-08-18T13:39:11.033 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-08-18T13:39:11.033 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-08-18T13:39:11.048 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 6faa90ce-e1a9-481a-8f60-55bc2ed16004 0 2024-08-18T13:39:11.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.148+0000 7fd1fdb02700 1 Processor -- start 2024-08-18T13:39:11.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.148+0000 7fd1fdb02700 1 -- start start 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd1f80c6b90 0x7fd1f80c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1f80ae190 0x7fd1f80ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f80c4520 con 0x7fd1f815d0d0 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f80c4800 con 0x7fd1f80c6b90 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f80c4ae0 con 0x7fd1f80ae190 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1f80ae190 0x7fd1f80ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1f80ae190 0x7fd1f80ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41590/0 (socket says 172.21.15.178:41590) 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 -- 172.21.15.178:0/1186877998 learned_addr learned my addr 172.21.15.178:0/1186877998 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:11.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1effff700 1 --2- 172.21.15.178:0/1186877998 >> v2:172.21.15.118:3300/0 conn(0x7fd1f80c6b90 0x7fd1f80c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 -- 172.21.15.178:0/1186877998 >> v2:172.21.15.118:3300/0 conn(0x7fd1f80c6b90 msgr2=0x7fd1f80c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 --2- 172.21.15.178:0/1186877998 >> v2:172.21.15.118:3300/0 conn(0x7fd1f80c6b90 0x7fd1f80c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 -- 172.21.15.178:0/1186877998 >> v1:172.21.15.118:6789/0 conn(0x7fd1f815d0d0 legacy=0x7fd1f815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1ef7fe700 1 -- 172.21.15.178:0/1186877998 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108675698 0 0) 0x7fd1f80c4520 con 0x7fd1f815d0d0 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 -- 172.21.15.178:0/1186877998 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80c4dc0 con 0x7fd1f80ae190 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fd301700 1 --2- 172.21.15.178:0/1186877998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f80ae190 0x7fd1f80ae5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80ad7f0 tx=0x7fd1f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ba38657ab0648a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1ef7fe700 1 -- 172.21.15.178:0/1186877998 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f000e070 con 0x7fd1f80ae190 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1ef7fe700 1 -- 172.21.15.178:0/1186877998 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f000bab0 con 0x7fd1f80ae190 2024-08-18T13:39:11.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1ef7fe700 1 -- 172.21.15.178:0/1186877998 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00092e0 con 0x7fd1f80ae190 2024-08-18T13:39:11.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1186877998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f80ae190 msgr2=0x7fd1f80ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 --2- 172.21.15.178:0/1186877998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f80ae190 0x7fd1f80ae5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80ad7f0 tx=0x7fd1f0009840 comp rx=0 tx=0).stop 2024-08-18T13:39:11.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1186877998 shutdown_connections 2024-08-18T13:39:11.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 --2- 172.21.15.178:0/1186877998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f80ae190 0x7fd1f80ae5f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 --2- 172.21.15.178:0/1186877998 >> v2:172.21.15.118:3300/0 conn(0x7fd1f80c6b90 0x7fd1f80c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1186877998 >> 172.21.15.178:0/1186877998 conn(0x7fd1f806d630 msgr2=0x7fd1f806da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1186877998 shutdown_connections 2024-08-18T13:39:11.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.152+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1186877998 wait complete. 2024-08-18T13:39:11.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 Processor -- start 2024-08-18T13:39:11.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 -- start start 2024-08-18T13:39:11.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1f80ae190 0x7fd1f8169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1f80c6b90 0x7fd1f8153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f815d0d0 0x7fd1f8069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f815a2d0 con 0x7fd1f80c6b90 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f81591c0 con 0x7fd1f815d0d0 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8159d30 con 0x7fd1f80ae190 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f815d0d0 0x7fd1f8069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f815d0d0 0x7fd1f8069e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41604/0 (socket says 172.21.15.178:41604) 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 -- 172.21.15.178:0/1184023850 learned_addr learned my addr 172.21.15.178:0/1184023850 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fcb00700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1f80ae190 0x7fd1f8169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1effff700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1f80c6b90 0x7fd1f8153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 -- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1f80ae190 msgr2=0x7fd1f8169f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1f80ae190 0x7fd1f8169f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 -- 172.21.15.178:0/1184023850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1f80c6b90 msgr2=0x7fd1f8153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1f80c6b90 0x7fd1f8153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 -- 172.21.15.178:0/1184023850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0009ae0 con 0x7fd1f815d0d0 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fd301700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f815d0d0 0x7fd1f8069e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80c4c20 tx=0x7fd1f0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1ecff9700 1 -- 172.21.15.178:0/1184023850 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00032f0 con 0x7fd1f815d0d0 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1ecff9700 1 -- 172.21.15.178:0/1184023850 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f0008d60 con 0x7fd1f815d0d0 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1ecff9700 1 -- 172.21.15.178:0/1184023850 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f001d670 con 0x7fd1f815d0d0 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f8159650 con 0x7fd1f815d0d0 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f8158890 con 0x7fd1f815d0d0 2024-08-18T13:39:11.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1ecff9700 1 -- 172.21.15.178:0/1184023850 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd1f0003040 con 0x7fd1f815d0d0 2024-08-18T13:39:11.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.156+0000 7fd1ecff9700 1 -- 172.21.15.178:0/1184023850 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd1f00277a0 con 0x7fd1f815d0d0 2024-08-18T13:39:11.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.160+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f8071410 con 0x7fd1f815d0d0 2024-08-18T13:39:11.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.160+0000 7fd1ecff9700 1 -- 172.21.15.178:0/1184023850 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fd1f002c060 con 0x7fd1f815d0d0 2024-08-18T13:39:11.202 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:39:11.205 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:39:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.240+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6faa90ce-e1a9-481a-8f60-55bc2ed16004", "id": 0} v 0) v1 -- 0x7fd1f8159300 con 0x7fd1f815d0d0 2024-08-18T13:39:11.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.244+0000 7fd1ecff9700 1 -- 172.21.15.178:0/1184023850 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6faa90ce-e1a9-481a-8f60-55bc2ed16004", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd1f0022070 con 0x7fd1f815d0d0 2024-08-18T13:39:11.252 INFO:teuthology.orchestra.run.smithi178.stdout:0 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f815d0d0 msgr2=0x7fd1f8069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f815d0d0 0x7fd1f8069e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80c4c20 tx=0x7fd1f0018000 comp rx=0 tx=0).stop 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 shutdown_connections 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f815d0d0 0x7fd1f8069e10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd1f80c6b90 0x7fd1f8153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 --2- 172.21.15.178:0/1184023850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1f80ae190 0x7fd1f8169f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 >> 172.21.15.178:0/1184023850 conn(0x7fd1f806d630 msgr2=0x7fd1f814e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 shutdown_connections 2024-08-18T13:39:11.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.248+0000 7fd1fdb02700 1 -- 172.21.15.178:0/1184023850 wait complete. 2024-08-18T13:39:11.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 9809a5fe-2ee4-4e13-95b8-35e09a43fa61 1 2024-08-18T13:39:11.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.412+0000 7fc91a010700 1 Processor -- start 2024-08-18T13:39:11.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.412+0000 7fc91a010700 1 -- start start 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc914069f90 0x7fc914069260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc914068dd0 con 0x7fc91414c450 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc914152ab0 con 0x7fc91406c3d0 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc914152d50 con 0x7fc914069f90 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91900e700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc91406c3d0 0x7fc91414bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:57878/0 (socket says 172.21.15.178:57878) 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91900e700 1 -- 172.21.15.178:0/2825055827 learned_addr learned my addr 172.21.15.178:0/2825055827 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91880d700 1 --2- 172.21.15.178:0/2825055827 >> v2:172.21.15.118:3300/0 conn(0x7fc914069f90 0x7fc914069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398916315 0 0) 0x7fc914152ab0 con 0x7fc91406c3d0 2024-08-18T13:39:11.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc900003570 con 0x7fc91406c3d0 2024-08-18T13:39:11.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044796032 0 0) 0x7fc914068dd0 con 0x7fc91414c450 2024-08-18T13:39:11.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc914152ab0 con 0x7fc91414c450 2024-08-18T13:39:11.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2320153867 0 0) 0x7fc900003570 con 0x7fc91406c3d0 2024-08-18T13:39:11.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 >> v2:172.21.15.118:3300/0 conn(0x7fc914069f90 msgr2=0x7fc914069260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:11.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 --2- 172.21.15.178:0/2825055827 >> v2:172.21.15.118:3300/0 conn(0x7fc914069f90 0x7fc914069260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 >> v1:172.21.15.118:6789/0 conn(0x7fc91414c450 legacy=0x7fc91415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140684e0 con 0x7fc91406c3d0 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fc9100013f0 con 0x7fc91406c3d0 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 >> v1:172.21.15.178:6789/0 conn(0x7fc91406c3d0 legacy=0x7fc91414bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900003c40 0x7fc900004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc900004810 0x7fc900004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc900005210 0x7fc90000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914152d50 con 0x7fc900003c40 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90000c130 con 0x7fc900004810 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90000c2c0 con 0x7fc900005210 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91880d700 1 --2- 172.21.15.178:0/2825055827 >> v2:172.21.15.118:3300/0 conn(0x7fc914069f90 0x7fc914069260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91880d700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc900004810 0x7fc900004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91900e700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900003c40 0x7fc900004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc900005210 0x7fc90000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 -- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc900004810 msgr2=0x7fc900004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc900004810 0x7fc900004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 -- 172.21.15.178:0/2825055827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900003c40 msgr2=0x7fc900004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900003c40 0x7fc900004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 -- 172.21.15.178:0/2825055827 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900003570 con 0x7fc900005210 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91880d700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc900004810 0x7fc900004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 -- 172.21.15.178:0/2825055827 reap_dead start 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91980f700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc900005210 0x7fc90000ba60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00e440 tx=0x7fc90c001be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e61815d9562819e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c001ef0 con 0x7fc900005210 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c00ecd0 con 0x7fc900005210 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc90bfff700 1 -- 172.21.15.178:0/2825055827 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c014630 con 0x7fc900005210 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 -- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc900005210 msgr2=0x7fc90000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc900005210 0x7fc90000ba60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00e440 tx=0x7fc90c001be0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 -- 172.21.15.178:0/2825055827 shutdown_connections 2024-08-18T13:39:11.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 --2- 172.21.15.178:0/2825055827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc900005210 0x7fc90000ba60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.416+0000 7fc91a010700 1 -- 172.21.15.178:0/2825055827 >> 172.21.15.178:0/2825055827 conn(0x7fc9140c5e00 msgr2=0x7fc9140c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 -- 172.21.15.178:0/2825055827 shutdown_connections 2024-08-18T13:39:11.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 -- 172.21.15.178:0/2825055827 wait complete. 2024-08-18T13:39:11.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 Processor -- start 2024-08-18T13:39:11.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 -- start start 2024-08-18T13:39:11.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900005210 0x7fc914145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc914151680 0x7fc914160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9141611f0 0x7fc914144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141651a0 con 0x7fc900005210 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9140c3f50 con 0x7fc914151680 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91a010700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc914164d10 con 0x7fc9141611f0 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91880d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc914151680 0x7fc914160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91980f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9141611f0 0x7fc914144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91880d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc914151680 0x7fc914160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41622/0 (socket says 172.21.15.178:41622) 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91880d700 1 -- 172.21.15.178:0/2340872698 learned_addr learned my addr 172.21.15.178:0/2340872698 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91900e700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900005210 0x7fc914145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91980f700 1 -- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc914151680 msgr2=0x7fc914160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91980f700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc914151680 0x7fc914160cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91980f700 1 -- 172.21.15.178:0/2340872698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900005210 msgr2=0x7fc914145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91980f700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900005210 0x7fc914145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91980f700 1 -- 172.21.15.178:0/2340872698 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9141567b0 con 0x7fc9141611f0 2024-08-18T13:39:11.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91900e700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900005210 0x7fc914145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.420+0000 7fc91880d700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc914151680 0x7fc914160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc91980f700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9141611f0 0x7fc914144280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc914070400 tx=0x7fc90c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc9097fa700 1 -- 172.21.15.178:0/2340872698 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c001ef0 con 0x7fc9141611f0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc9097fa700 1 -- 172.21.15.178:0/2340872698 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c014e20 con 0x7fc9141611f0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9140c3c70 con 0x7fc9141611f0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc9097fa700 1 -- 172.21.15.178:0/2340872698 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c013d20 con 0x7fc9141611f0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9141646c0 con 0x7fc9141611f0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc9097fa700 1 -- 172.21.15.178:0/2340872698 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc90c003030 con 0x7fc9141611f0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc9097fa700 1 -- 172.21.15.178:0/2340872698 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fc90c020bd0 con 0x7fc9141611f0 2024-08-18T13:39:11.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.424+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc914071410 con 0x7fc9141611f0 2024-08-18T13:39:11.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.428+0000 7fc9097fa700 1 -- 172.21.15.178:0/2340872698 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc90c020de0 con 0x7fc9141611f0 2024-08-18T13:39:11.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.504+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9809a5fe-2ee4-4e13-95b8-35e09a43fa61", "id": 1} v 0) v1 -- 0x7fc914071410 con 0x7fc9141611f0 2024-08-18T13:39:11.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc9097fa700 1 -- 172.21.15.178:0/2340872698 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9809a5fe-2ee4-4e13-95b8-35e09a43fa61", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc90c035480 con 0x7fc9141611f0 2024-08-18T13:39:11.517 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-08-18T13:39:11.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9141611f0 msgr2=0x7fc914144280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9141611f0 0x7fc914144280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc914070400 tx=0x7fc90c018000 comp rx=0 tx=0).stop 2024-08-18T13:39:11.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 shutdown_connections 2024-08-18T13:39:11.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9141611f0 0x7fc914144280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc914151680 0x7fc914160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 --2- 172.21.15.178:0/2340872698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc900005210 0x7fc914145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 >> 172.21.15.178:0/2340872698 conn(0x7fc9140c5e00 msgr2=0x7fc914067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 shutdown_connections 2024-08-18T13:39:11.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.512+0000 7fc91a010700 1 -- 172.21.15.178:0/2340872698 wait complete. 2024-08-18T13:39:11.529 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new d1dea5ed-5651-486c-99fa-f82ca3216928 2 2024-08-18T13:39:11.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a56b39700 1 Processor -- start 2024-08-18T13:39:11.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a56b39700 1 -- start start 2024-08-18T13:39:11.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a56b39700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7a500c1b00 0x7f7a50149680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a56b39700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a50149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a56b39700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a50067710 con 0x7f7a500c1ed0 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a56b39700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a501540b0 con 0x7f7a500c1b00 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a56b39700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50153dd0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a50149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a550d6700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7a500c1ed0 0x7f7a50067ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:57880/0 (socket says 172.21.15.178:57880) 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a550d6700 1 -- 172.21.15.178:0/3966940520 learned_addr learned my addr 172.21.15.178:0/3966940520 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a548d5700 1 --2- 172.21.15.178:0/3966940520 >> v2:172.21.15.118:3300/0 conn(0x7f7a500c1b00 0x7f7a50149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4f7fe700 1 -- 172.21.15.178:0/3966940520 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245089413 0 0) 0x7f7a50067710 con 0x7f7a500c1ed0 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4f7fe700 1 -- 172.21.15.178:0/3966940520 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a38003570 con 0x7f7a500c1ed0 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4ffff700 1 -- 172.21.15.178:0/3966940520 >> v2:172.21.15.118:3300/0 conn(0x7f7a500c1b00 msgr2=0x7f7a50149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:11.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4ffff700 1 --2- 172.21.15.178:0/3966940520 >> v2:172.21.15.118:3300/0 conn(0x7f7a500c1b00 0x7f7a50149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4ffff700 1 -- 172.21.15.178:0/3966940520 >> v1:172.21.15.178:6789/0 conn(0x7f7a500c1ed0 legacy=0x7f7a50067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4ffff700 1 -- 172.21.15.178:0/3966940520 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500c4530 con 0x7f7a5005e6b0 2024-08-18T13:39:11.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.628+0000 7f7a4ffff700 1 --2- 172.21.15.178:0/3966940520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a50149bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a440098e0 tx=0x7f7a440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfc62a0e770484b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a4f7fe700 1 -- 172.21.15.178:0/3966940520 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44011630 con 0x7f7a5005e6b0 2024-08-18T13:39:11.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a4f7fe700 1 -- 172.21.15.178:0/3966940520 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a44011790 con 0x7f7a5005e6b0 2024-08-18T13:39:11.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a4f7fe700 1 -- 172.21.15.178:0/3966940520 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44011b30 con 0x7f7a5005e6b0 2024-08-18T13:39:11.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 -- 172.21.15.178:0/3966940520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 msgr2=0x7f7a50149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 --2- 172.21.15.178:0/3966940520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a50149bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a440098e0 tx=0x7f7a440091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 -- 172.21.15.178:0/3966940520 shutdown_connections 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 --2- 172.21.15.178:0/3966940520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a50149bc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 --2- 172.21.15.178:0/3966940520 >> v2:172.21.15.118:3300/0 conn(0x7f7a500c1b00 0x7f7a50149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 -- 172.21.15.178:0/3966940520 >> 172.21.15.178:0/3966940520 conn(0x7f7a500c8c90 msgr2=0x7f7a500c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 -- 172.21.15.178:0/3966940520 shutdown_connections 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 -- 172.21.15.178:0/3966940520 wait complete. 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 Processor -- start 2024-08-18T13:39:11.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.632+0000 7f7a56b39700 1 -- start start 2024-08-18T13:39:11.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a5014a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a500c1b00 0x7f7a501568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a500c1ed0 0x7f7a50156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5014de30 con 0x7f7a500c1ed0 2024-08-18T13:39:11.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a501637c0 con 0x7f7a500c1b00 2024-08-18T13:39:11.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50163ca0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a500c1b00 0x7f7a501568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a548d5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a5014a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a500c1b00 0x7f7a501568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41636/0 (socket says 172.21.15.178:41636) 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4ffff700 1 -- 172.21.15.178:0/498048728 learned_addr learned my addr 172.21.15.178:0/498048728 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a550d6700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a500c1ed0 0x7f7a50156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a548d5700 1 -- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a500c1b00 msgr2=0x7f7a501568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a548d5700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a500c1b00 0x7f7a501568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a548d5700 1 -- 172.21.15.178:0/498048728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a500c1ed0 msgr2=0x7f7a50156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a548d5700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a500c1ed0 0x7f7a50156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a548d5700 1 -- 172.21.15.178:0/498048728 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44009420 con 0x7f7a5005e6b0 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4ffff700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a500c1b00 0x7f7a501568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a550d6700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a500c1ed0 0x7f7a50156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:11.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a548d5700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a5014a340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ea80 tx=0x7f7a400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4cff9700 1 -- 172.21.15.178:0/498048728 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400095d0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4cff9700 1 -- 172.21.15.178:0/498048728 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000c7f0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a50156460 con 0x7f7a5005e6b0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4cff9700 1 -- 172.21.15.178:0/498048728 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000cbb0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a50155b30 con 0x7f7a5005e6b0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4cff9700 1 -- 172.21.15.178:0/498048728 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7a40005a00 con 0x7f7a5005e6b0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a4cff9700 1 -- 172.21.15.178:0/498048728 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f7a40006810 con 0x7f7a5005e6b0 2024-08-18T13:39:11.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.636+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a500c60e0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.640+0000 7f7a4cff9700 1 -- 172.21.15.178:0/498048728 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7a500c60e0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.698 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:11.692+0000 7fdfd8c41000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-18T13:39:11.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.732+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d1dea5ed-5651-486c-99fa-f82ca3216928", "id": 2} v 0) v1 -- 0x7f7a500648f0 con 0x7f7a5005e6b0 2024-08-18T13:39:11.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.736+0000 7f7a4cff9700 1 -- 172.21.15.178:0/498048728 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1dea5ed-5651-486c-99fa-f82ca3216928", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7a40029020 con 0x7f7a5005e6b0 2024-08-18T13:39:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:2 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.736+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 msgr2=0x7f7a5014a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.736+0000 7f7a56b39700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a5014a340 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ea80 tx=0x7f7a400091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.740+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 shutdown_connections 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.740+0000 7f7a56b39700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a500c1ed0 0x7f7a50156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.740+0000 7f7a56b39700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a500c1b00 0x7f7a501568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.740+0000 7f7a56b39700 1 --2- 172.21.15.178:0/498048728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a5005e6b0 0x7f7a5014a340 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.740+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 >> 172.21.15.178:0/498048728 conn(0x7f7a500c8c90 msgr2=0x7f7a50151ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.740+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 shutdown_connections 2024-08-18T13:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.740+0000 7f7a56b39700 1 -- 172.21.15.178:0/498048728 wait complete. 2024-08-18T13:39:11.746 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:11.739+0000 7f3a305dc000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-18T13:39:11.758 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:11.752+0000 7fdfd8c41000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-18T13:39:11.759 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de 3 2024-08-18T13:39:11.802 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:11.799+0000 7f3a305dc000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-18T13:39:11.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b2a36700 1 Processor -- start 2024-08-18T13:39:11.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b2a36700 1 -- start start 2024-08-18T13:39:11.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b2a36700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48ac0c3350 0x7f48ac14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b2a36700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac0c3de0 con 0x7f48ac0c2f80 2024-08-18T13:39:11.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b2a36700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac06d620 con 0x7f48ac14be20 2024-08-18T13:39:11.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b2a36700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ac0bfb90 con 0x7f48ac0c3350 2024-08-18T13:39:11.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b1233700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48ac0c3350 0x7f48ac14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b1233700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48ac0c3350 0x7f48ac14b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41652/0 (socket says 172.21.15.178:41652) 2024-08-18T13:39:11.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b1233700 1 -- 172.21.15.178:0/1154498334 learned_addr learned my addr 172.21.15.178:0/1154498334 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:11.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b2235700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f48ac14be20 0x7f48ac15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:57884/0 (socket says 172.21.15.178:57884) 2024-08-18T13:39:11.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b0a32700 1 -- 172.21.15.178:0/1154498334 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829625601 0 0) 0x7f48ac06d620 con 0x7f48ac14be20 2024-08-18T13:39:11.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b0a32700 1 -- 172.21.15.178:0/1154498334 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4898003570 con 0x7f48ac14be20 2024-08-18T13:39:11.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b0a32700 1 -- 172.21.15.178:0/1154498334 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282890231 0 0) 0x7f48ac0c3de0 con 0x7f48ac0c2f80 2024-08-18T13:39:11.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b0a32700 1 -- 172.21.15.178:0/1154498334 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48ac06d620 con 0x7f48ac0c2f80 2024-08-18T13:39:11.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b1233700 1 -- 172.21.15.178:0/1154498334 >> v1:172.21.15.178:6789/0 conn(0x7f48ac14be20 legacy=0x7f48ac15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b1233700 1 -- 172.21.15.178:0/1154498334 >> v1:172.21.15.118:6789/0 conn(0x7f48ac0c2f80 legacy=0x7f48ac0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b1233700 1 -- 172.21.15.178:0/1154498334 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac0c4980 con 0x7f48ac0c3350 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.852+0000 7f48b1233700 1 --2- 172.21.15.178:0/1154498334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c3350 0x7f48ac14b8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f489c0099b0 tx=0x7f489c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f4689fb2339eb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b0a32700 1 -- 172.21.15.178:0/1154498334 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00b680 con 0x7f48ac0c3350 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b0a32700 1 -- 172.21.15.178:0/1154498334 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c00b7e0 con 0x7f48ac0c3350 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b0a32700 1 -- 172.21.15.178:0/1154498334 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00bb80 con 0x7f48ac0c3350 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 -- 172.21.15.178:0/1154498334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c3350 msgr2=0x7f48ac14b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 --2- 172.21.15.178:0/1154498334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c3350 0x7f48ac14b8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f489c0099b0 tx=0x7f489c009250 comp rx=0 tx=0).stop 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 -- 172.21.15.178:0/1154498334 shutdown_connections 2024-08-18T13:39:11.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 --2- 172.21.15.178:0/1154498334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c3350 0x7f48ac14b8e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 -- 172.21.15.178:0/1154498334 >> 172.21.15.178:0/1154498334 conn(0x7f48ac06c750 msgr2=0x7f48ac0c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 -- 172.21.15.178:0/1154498334 shutdown_connections 2024-08-18T13:39:11.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 -- 172.21.15.178:0/1154498334 wait complete. 2024-08-18T13:39:11.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 Processor -- start 2024-08-18T13:39:11.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.856+0000 7f48b2a36700 1 -- start start 2024-08-18T13:39:11.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48ac0c2f80 0x7f48ac0c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48ac14be20 0x7f48ac1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c6480 0x7f48ac150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac161ca0 con 0x7f48ac0c2f80 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac150ef0 con 0x7f48ac0c6480 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac14fc50 con 0x7f48ac14be20 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48ac14be20 0x7f48ac1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2235700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c6480 0x7f48ac150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48ac14be20 0x7f48ac1500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:53330/0 (socket says 172.21.15.178:53330) 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 -- 172.21.15.178:0/219187542 learned_addr learned my addr 172.21.15.178:0/219187542 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2235700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c6480 0x7f48ac150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41666/0 (socket says 172.21.15.178:41666) 2024-08-18T13:39:11.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1a34700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48ac0c2f80 0x7f48ac0c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 -- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c6480 msgr2=0x7f48ac150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c6480 0x7f48ac150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 -- 172.21.15.178:0/219187542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48ac0c2f80 msgr2=0x7f48ac0c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48ac0c2f80 0x7f48ac0c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 -- 172.21.15.178:0/219187542 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0094f0 con 0x7f48ac14be20 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2235700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c6480 0x7f48ac150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b1233700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48ac14be20 0x7f48ac1500e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48ac06d7b0 tx=0x7f489c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48a1ffb700 1 -- 172.21.15.178:0/219187542 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00b950 con 0x7f48ac14be20 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48a1ffb700 1 -- 172.21.15.178:0/219187542 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c00bad0 con 0x7f48ac14be20 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac161120 con 0x7f48ac14be20 2024-08-18T13:39:11.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48a1ffb700 1 -- 172.21.15.178:0/219187542 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0276a0 con 0x7f48ac14be20 2024-08-18T13:39:11.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac1616e0 con 0x7f48ac14be20 2024-08-18T13:39:11.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48a1ffb700 1 -- 172.21.15.178:0/219187542 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f489c027a50 con 0x7f48ac14be20 2024-08-18T13:39:11.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.860+0000 7f48a1ffb700 1 -- 172.21.15.178:0/219187542 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f489c027d00 con 0x7f48ac14be20 2024-08-18T13:39:11.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.864+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac14fa20 con 0x7f48ac14be20 2024-08-18T13:39:11.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.864+0000 7f48a1ffb700 1 -- 172.21.15.178:0/219187542 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f489c0477a0 con 0x7f48ac14be20 2024-08-18T13:39:11.875 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:11.868+0000 7fdfd8c41000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-18T13:39:11.916 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:11.911+0000 7f3a305dc000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-18T13:39:11.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.940+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de", "id": 3} v 0) v1 -- 0x7f48ac14fdc0 con 0x7f48ac14be20 2024-08-18T13:39:11.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48a1ffb700 1 -- 172.21.15.178:0/219187542 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f489c036070 con 0x7f48ac14be20 2024-08-18T13:39:11.951 INFO:teuthology.orchestra.run.smithi178.stdout:3 2024-08-18T13:39:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48ac14be20 msgr2=0x7f48ac1500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48ac14be20 0x7f48ac1500e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48ac06d7b0 tx=0x7f489c017000 comp rx=0 tx=0).stop 2024-08-18T13:39:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 shutdown_connections 2024-08-18T13:39:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac0c6480 0x7f48ac150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48ac14be20 0x7f48ac1500e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 --2- 172.21.15.178:0/219187542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48ac0c2f80 0x7f48ac0c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 >> 172.21.15.178:0/219187542 conn(0x7f48ac06c750 msgr2=0x7f48ac0c4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 shutdown_connections 2024-08-18T13:39:11.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:11.944+0000 7f48b2a36700 1 -- 172.21.15.178:0/219187542 wait complete. 2024-08-18T13:39:11.964 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 66ab9cf2-8285-49a5-b2ed-c3931c19a983 4 2024-08-18T13:39:12.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.060+0000 7ff4d23ff700 1 Processor -- start 2024-08-18T13:39:12.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.060+0000 7ff4d23ff700 1 -- start start 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4cc14bca0 0x7ff4cc0c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc0681f0 0x7ff4cc0c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc06cc70 con 0x7ff4cc0688a0 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4cc067dd0 con 0x7ff4cc14bca0 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc068630 con 0x7ff4cc0681f0 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4cbfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4cc14bca0 0x7ff4cc0c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d099c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc0681f0 0x7ff4cc0c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4cbfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4cc14bca0 0x7ff4cc0c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41682/0 (socket says 172.21.15.178:41682) 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4cbfff700 1 -- 172.21.15.178:0/1762966505 learned_addr learned my addr 172.21.15.178:0/1762966505 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d099c700 1 -- 172.21.15.178:0/1762966505 >> v2:172.21.15.178:3300/0 conn(0x7ff4cc14bca0 msgr2=0x7ff4cc0c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:12.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d099c700 1 --2- 172.21.15.178:0/1762966505 >> v2:172.21.15.178:3300/0 conn(0x7ff4cc14bca0 0x7ff4cc0c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d099c700 1 -- 172.21.15.178:0/1762966505 >> v1:172.21.15.118:6789/0 conn(0x7ff4cc0688a0 legacy=0x7ff4cc06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d099c700 1 -- 172.21.15.178:0/1762966505 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc067470 con 0x7ff4cc0681f0 2024-08-18T13:39:12.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4cbfff700 1 --2- 172.21.15.178:0/1762966505 >> v2:172.21.15.178:3300/0 conn(0x7ff4cc14bca0 0x7ff4cc0c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:12.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d099c700 1 --2- 172.21.15.178:0/1762966505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc0681f0 0x7ff4cc0c06e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009d50 tx=0x7ff4c4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07a06fcd4dbec49 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4caffd700 1 -- 172.21.15.178:0/1762966505 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400cd00 con 0x7ff4cc0681f0 2024-08-18T13:39:12.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4caffd700 1 -- 172.21.15.178:0/1762966505 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c400ce60 con 0x7ff4cc0681f0 2024-08-18T13:39:12.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4caffd700 1 -- 172.21.15.178:0/1762966505 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4011650 con 0x7ff4cc0681f0 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- 172.21.15.178:0/1762966505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc0681f0 msgr2=0x7ff4cc0c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 --2- 172.21.15.178:0/1762966505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc0681f0 0x7ff4cc0c06e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009d50 tx=0x7ff4c4009780 comp rx=0 tx=0).stop 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- 172.21.15.178:0/1762966505 shutdown_connections 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 --2- 172.21.15.178:0/1762966505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc0681f0 0x7ff4cc0c06e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 --2- 172.21.15.178:0/1762966505 >> v2:172.21.15.178:3300/0 conn(0x7ff4cc14bca0 0x7ff4cc0c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- 172.21.15.178:0/1762966505 >> 172.21.15.178:0/1762966505 conn(0x7ff4cc0c8d60 msgr2=0x7ff4cc0c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- 172.21.15.178:0/1762966505 shutdown_connections 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.064+0000 7ff4d23ff700 1 -- 172.21.15.178:0/1762966505 wait complete. 2024-08-18T13:39:12.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 Processor -- start 2024-08-18T13:39:12.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 -- start start 2024-08-18T13:39:12.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc0681f0 0x7ff4cc150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0688a0 0x7ff4cc156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc14bca0 0x7ff4cc157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1611d0 con 0x7ff4cc0688a0 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1647b0 con 0x7ff4cc0681f0 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc161430 con 0x7ff4cc14bca0 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc0681f0 0x7ff4cc150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d099c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc14bca0 0x7ff4cc157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc0681f0 0x7ff4cc150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41694/0 (socket says 172.21.15.178:41694) 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4cbfff700 1 -- 172.21.15.178:0/3641051710 learned_addr learned my addr 172.21.15.178:0/3641051710 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4cb7fe700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0688a0 0x7ff4cc156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d099c700 1 -- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc0681f0 msgr2=0x7ff4cc150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d099c700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc0681f0 0x7ff4cc150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d099c700 1 -- 172.21.15.178:0/3641051710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0688a0 msgr2=0x7ff4cc156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d099c700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0688a0 0x7ff4cc156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d099c700 1 -- 172.21.15.178:0/3641051710 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc0094f0 con 0x7ff4cc14bca0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4cbfff700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc0681f0 0x7ff4cc150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d099c700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc14bca0 0x7ff4cc157510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4017000 tx=0x7ff4c401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4b3fff700 1 -- 172.21.15.178:0/3641051710 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4021070 con 0x7ff4cc14bca0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c400d040 con 0x7ff4cc14bca0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4b3fff700 1 -- 172.21.15.178:0/3641051710 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c4011e90 con 0x7ff4cc14bca0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc164d30 con 0x7ff4cc14bca0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4b3fff700 1 -- 172.21.15.178:0/3641051710 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400cd00 con 0x7ff4cc14bca0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4b3fff700 1 -- 172.21.15.178:0/3641051710 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff4c4003040 con 0x7ff4cc14bca0 2024-08-18T13:39:12.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.068+0000 7ff4b3fff700 1 -- 172.21.15.178:0/3641051710 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7ff4c401aaa0 con 0x7ff4cc14bca0 2024-08-18T13:39:12.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.072+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc0648f0 con 0x7ff4cc14bca0 2024-08-18T13:39:12.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.072+0000 7ff4b3fff700 1 -- 172.21.15.178:0/3641051710 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff4c4033060 con 0x7ff4cc14bca0 2024-08-18T13:39:12.140 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:12.132+0000 7fdfd8c41000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-18T13:39:12.155 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:12.151+0000 7f3a305dc000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-18T13:39:12.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.156+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "66ab9cf2-8285-49a5-b2ed-c3931c19a983", "id": 4} v 0) v1 -- 0x7ff4cc06ef10 con 0x7ff4cc14bca0 2024-08-18T13:39:12.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.160+0000 7ff4b3fff700 1 -- 172.21.15.178:0/3641051710 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "66ab9cf2-8285-49a5-b2ed-c3931c19a983", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff4c402a070 con 0x7ff4cc14bca0 2024-08-18T13:39:12.168 INFO:teuthology.orchestra.run.smithi178.stdout:4 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc14bca0 msgr2=0x7ff4cc157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc14bca0 0x7ff4cc157510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4017000 tx=0x7ff4c401a000 comp rx=0 tx=0).stop 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 shutdown_connections 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4cc14bca0 0x7ff4cc157510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4cc0688a0 0x7ff4cc156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 --2- 172.21.15.178:0/3641051710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4cc0681f0 0x7ff4cc150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 >> 172.21.15.178:0/3641051710 conn(0x7ff4cc0c8d60 msgr2=0x7ff4cc0c6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 shutdown_connections 2024-08-18T13:39:12.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.164+0000 7ff4d23ff700 1 -- 172.21.15.178:0/3641051710 wait complete. 2024-08-18T13:39:12.180 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 45763a0e-ecdc-4810-bc05-f1a2f14f2791 5 2024-08-18T13:39:12.197 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:12.192+0000 7fdfd8c41000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-18T13:39:12.208 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:12.203+0000 7f3a305dc000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-18T13:39:12.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be8596700 1 Processor -- start 2024-08-18T13:39:12.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be8596700 1 -- start start 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be8596700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2be006ece0 0x7f2be0152970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be8596700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2be00c40d0 0x7f2be014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be8596700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be00c02d0 con 0x7f2be00bf970 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be8596700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be00c0d50 con 0x7f2be006ece0 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be8596700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be0152560 con 0x7f2be00c40d0 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6b33700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2be00c40d0 0x7f2be014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6b33700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2be00c40d0 0x7f2be014fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41702/0 (socket says 172.21.15.178:41702) 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6b33700 1 -- 172.21.15.178:0/593069197 learned_addr learned my addr 172.21.15.178:0/593069197 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6332700 1 --2- 172.21.15.178:0/593069197 >> v2:172.21.15.118:3300/0 conn(0x7f2be006ece0 0x7f2be0152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6b33700 1 -- 172.21.15.178:0/593069197 >> v2:172.21.15.118:3300/0 conn(0x7f2be006ece0 msgr2=0x7f2be0152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6b33700 1 --2- 172.21.15.178:0/593069197 >> v2:172.21.15.118:3300/0 conn(0x7f2be006ece0 0x7f2be0152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6b33700 1 -- 172.21.15.178:0/593069197 >> v1:172.21.15.118:6789/0 conn(0x7f2be00bf970 legacy=0x7f2be014f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.276+0000 7f2be6b33700 1 -- 172.21.15.178:0/593069197 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0155b80 con 0x7f2be00c40d0 2024-08-18T13:39:12.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be6b33700 1 --2- 172.21.15.178:0/593069197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be014fb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2be01526a0 tx=0x7f2bd8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae15ab12e8d0ec40 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be5330700 1 -- 172.21.15.178:0/593069197 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800e070 con 0x7f2be00c40d0 2024-08-18T13:39:12.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be5330700 1 -- 172.21.15.178:0/593069197 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd800b9f0 con 0x7f2be00c40d0 2024-08-18T13:39:12.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be5330700 1 -- 172.21.15.178:0/593069197 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800bdb0 con 0x7f2be00c40d0 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 -- 172.21.15.178:0/593069197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 msgr2=0x7f2be014fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 --2- 172.21.15.178:0/593069197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be014fb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2be01526a0 tx=0x7f2bd8009840 comp rx=0 tx=0).stop 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 -- 172.21.15.178:0/593069197 shutdown_connections 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 --2- 172.21.15.178:0/593069197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be014fb30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 --2- 172.21.15.178:0/593069197 >> v2:172.21.15.118:3300/0 conn(0x7f2be006ece0 0x7f2be0152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 -- 172.21.15.178:0/593069197 >> 172.21.15.178:0/593069197 conn(0x7f2be00c96e0 msgr2=0x7f2be006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 -- 172.21.15.178:0/593069197 shutdown_connections 2024-08-18T13:39:12.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 -- 172.21.15.178:0/593069197 wait complete. 2024-08-18T13:39:12.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 Processor -- start 2024-08-18T13:39:12.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.280+0000 7f2be8596700 1 -- start start 2024-08-18T13:39:12.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006ece0 0x7f2be0164320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be00bf970 0x7f2be0166e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be0150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00b6c40 con 0x7f2be00bf970 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00c8260 con 0x7f2be00c40d0 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00be990 con 0x7f2be006ece0 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6332700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006ece0 0x7f2be0164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6b33700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be0150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6332700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006ece0 0x7f2be0164320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:53364/0 (socket says 172.21.15.178:53364) 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6332700 1 -- 172.21.15.178:0/756496607 learned_addr learned my addr 172.21.15.178:0/756496607 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be5b31700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be00bf970 0x7f2be0166e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6b33700 1 -- 172.21.15.178:0/756496607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006ece0 msgr2=0x7f2be0164320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6b33700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006ece0 0x7f2be0164320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6b33700 1 -- 172.21.15.178:0/756496607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be00bf970 msgr2=0x7f2be0166e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6b33700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be00bf970 0x7f2be0166e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6b33700 1 -- 172.21.15.178:0/756496607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8009ae0 con 0x7f2be00c40d0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be6b33700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be0150ac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8014000 tx=0x7f2bd8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2bd67fc700 1 -- 172.21.15.178:0/756496607 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800bee0 con 0x7f2be00c40d0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2bd67fc700 1 -- 172.21.15.178:0/756496607 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd80032f0 con 0x7f2be00c40d0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be00c7bb0 con 0x7f2be00c40d0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2bd67fc700 1 -- 172.21.15.178:0/756496607 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801d420 con 0x7f2be00c40d0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be00c8470 con 0x7f2be00c40d0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2bd67fc700 1 -- 172.21.15.178:0/756496607 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f2bd801d850 con 0x7f2be00c40d0 2024-08-18T13:39:12.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2bd67fc700 1 -- 172.21.15.178:0/756496607 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f2bd8025970 con 0x7f2be00c40d0 2024-08-18T13:39:12.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.284+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be00beb00 con 0x7f2be00c40d0 2024-08-18T13:39:12.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.288+0000 7f2bd67fc700 1 -- 172.21.15.178:0/756496607 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f2bd80447a0 con 0x7f2be00c40d0 2024-08-18T13:39:12.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.376+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "45763a0e-ecdc-4810-bc05-f1a2f14f2791", "id": 5} v 0) v1 -- 0x7f2be00648f0 con 0x7f2be00c40d0 2024-08-18T13:39:12.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2bd67fc700 1 -- 172.21.15.178:0/756496607 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "45763a0e-ecdc-4810-bc05-f1a2f14f2791", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2bd80330b0 con 0x7f2be00c40d0 2024-08-18T13:39:12.387 INFO:teuthology.orchestra.run.smithi178.stdout:5 2024-08-18T13:39:12.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 msgr2=0x7f2be0150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be0150ac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8014000 tx=0x7f2bd8017000 comp rx=0 tx=0).stop 2024-08-18T13:39:12.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 shutdown_connections 2024-08-18T13:39:12.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00c40d0 0x7f2be0150ac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be00bf970 0x7f2be0166e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 --2- 172.21.15.178:0/756496607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006ece0 0x7f2be0164320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 >> 172.21.15.178:0/756496607 conn(0x7f2be00c96e0 msgr2=0x7f2be0152800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 shutdown_connections 2024-08-18T13:39:12.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.380+0000 7f2be8596700 1 -- 172.21.15.178:0/756496607 wait complete. 2024-08-18T13:39:12.394 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:12.388+0000 7fdfd8c41000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-18T13:39:12.399 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new e5380e12-4fba-4df2-9efb-c0e472e52c5b 6 2024-08-18T13:39:12.408 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:12.403+0000 7f3a305dc000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-18T13:39:12.453 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:12.444+0000 7fdfd8c41000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-18T13:39:12.475 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:12.471+0000 7f3a305dc000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-18T13:39:12.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.492+0000 7fb9e4d5c700 1 Processor -- start 2024-08-18T13:39:12.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.492+0000 7fb9e4d5c700 1 -- start start 2024-08-18T13:39:12.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.492+0000 7fb9e4d5c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb9e014f0d0 0x7fb9e0155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9e0156890 0x7fb9e00c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e0156300 con 0x7fb9e014f4a0 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00c2750 con 0x7fb9e014f0d0 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00c29f0 con 0x7fb9e0156890 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9e0156890 0x7fb9e00c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9e0156890 0x7fb9e00c1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41718/0 (socket says 172.21.15.178:41718) 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 -- 172.21.15.178:0/2699398426 learned_addr learned my addr 172.21.15.178:0/2699398426 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9de59c700 1 --2- 172.21.15.178:0/2699398426 >> v2:172.21.15.118:3300/0 conn(0x7fb9e014f0d0 0x7fb9e0155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 -- 172.21.15.178:0/2699398426 >> v2:172.21.15.118:3300/0 conn(0x7fb9e014f0d0 msgr2=0x7fb9e0155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 --2- 172.21.15.178:0/2699398426 >> v2:172.21.15.118:3300/0 conn(0x7fb9e014f0d0 0x7fb9e0155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 -- 172.21.15.178:0/2699398426 >> v1:172.21.15.118:6789/0 conn(0x7fb9e014f4a0 legacy=0x7fb9e00c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 -- 172.21.15.178:0/2699398426 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00c0c40 con 0x7fb9e0156890 2024-08-18T13:39:12.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9d7fff700 1 --2- 172.21.15.178:0/2699398426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e0156890 0x7fb9e00c1680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00c86f0 tx=0x7fb9c8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3fc750f16e43f54 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9ddd9b700 1 -- 172.21.15.178:0/2699398426 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800b860 con 0x7fb9e0156890 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9ddd9b700 1 -- 172.21.15.178:0/2699398426 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c800b9c0 con 0x7fb9e0156890 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9ddd9b700 1 -- 172.21.15.178:0/2699398426 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800bd60 con 0x7fb9e0156890 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/2699398426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e0156890 msgr2=0x7fb9e00c1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 --2- 172.21.15.178:0/2699398426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e0156890 0x7fb9e00c1680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00c86f0 tx=0x7fb9c8009250 comp rx=0 tx=0).stop 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/2699398426 shutdown_connections 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 --2- 172.21.15.178:0/2699398426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e0156890 0x7fb9e00c1680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 --2- 172.21.15.178:0/2699398426 >> v2:172.21.15.118:3300/0 conn(0x7fb9e014f0d0 0x7fb9e0155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/2699398426 >> 172.21.15.178:0/2699398426 conn(0x7fb9e00cc7a0 msgr2=0x7fb9e00c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/2699398426 shutdown_connections 2024-08-18T13:39:12.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/2699398426 wait complete. 2024-08-18T13:39:12.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 Processor -- start 2024-08-18T13:39:12.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.496+0000 7fb9e4d5c700 1 -- start start 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9e014f0d0 0x7fb9e0148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e014f4a0 0x7fb9e0153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e0156890 0x7fb9e0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0153d90 con 0x7fb9e0156890 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e014e5d0 con 0x7fb9e014f4a0 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01542a0 con 0x7fb9e014f0d0 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9e014f0d0 0x7fb9e0148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9d7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e014f4a0 0x7fb9e0153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9e014f0d0 0x7fb9e0148760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:53372/0 (socket says 172.21.15.178:53372) 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 -- 172.21.15.178:0/4030533949 learned_addr learned my addr 172.21.15.178:0/4030533949 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9ded9d700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e0156890 0x7fb9e0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 -- 172.21.15.178:0/4030533949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e014f4a0 msgr2=0x7fb9e0153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e014f4a0 0x7fb9e0153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 -- 172.21.15.178:0/4030533949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e0156890 msgr2=0x7fb9e0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e0156890 0x7fb9e0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 -- 172.21.15.178:0/4030533949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80094f0 con 0x7fb9e014f0d0 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9de59c700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9e014f0d0 0x7fb9e0148760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000d890 tx=0x7fb9d0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9d67fc700 1 -- 172.21.15.178:0/4030533949 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00125f0 con 0x7fb9e014f0d0 2024-08-18T13:39:12.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9d67fc700 1 -- 172.21.15.178:0/4030533949 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0012750 con 0x7fb9e014f0d0 2024-08-18T13:39:12.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9d67fc700 1 -- 172.21.15.178:0/4030533949 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0012af0 con 0x7fb9e014f0d0 2024-08-18T13:39:12.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e014e820 con 0x7fb9e014f0d0 2024-08-18T13:39:12.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e0154d00 con 0x7fb9e014f0d0 2024-08-18T13:39:12.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9d67fc700 1 -- 172.21.15.178:0/4030533949 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb9d001e6f0 con 0x7fb9e014f0d0 2024-08-18T13:39:12.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9d67fc700 1 -- 172.21.15.178:0/4030533949 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fb9d001e9a0 con 0x7fb9e014f0d0 2024-08-18T13:39:12.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.500+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e00642f0 con 0x7fb9e014f0d0 2024-08-18T13:39:12.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.504+0000 7fb9d67fc700 1 -- 172.21.15.178:0/4030533949 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fb9d001eb60 con 0x7fb9e014f0d0 2024-08-18T13:39:12.517 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:12.512+0000 7fdfd8c41000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-18T13:39:12.542 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:12.539+0000 7f3a305dc000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-18T13:39:12.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.580+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e5380e12-4fba-4df2-9efb-c0e472e52c5b", "id": 6} v 0) v1 -- 0x7fb9e00c1180 con 0x7fb9e014f0d0 2024-08-18T13:39:12.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.584+0000 7fb9d67fc700 1 -- 172.21.15.178:0/4030533949 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5380e12-4fba-4df2-9efb-c0e472e52c5b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb9d001ed00 con 0x7fb9e014f0d0 2024-08-18T13:39:12.592 INFO:teuthology.orchestra.run.smithi178.stdout:6 2024-08-18T13:39:12.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9e014f0d0 msgr2=0x7fb9e0148760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9e014f0d0 0x7fb9e0148760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000d890 tx=0x7fb9d0008040 comp rx=0 tx=0).stop 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 shutdown_connections 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb9e0156890 0x7fb9e0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9e014f4a0 0x7fb9e0153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 --2- 172.21.15.178:0/4030533949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9e014f0d0 0x7fb9e0148760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 >> 172.21.15.178:0/4030533949 conn(0x7fb9e00cc7a0 msgr2=0x7fb9e00c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 shutdown_connections 2024-08-18T13:39:12.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.588+0000 7fb9e4d5c700 1 -- 172.21.15.178:0/4030533949 wait complete. 2024-08-18T13:39:12.605 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 8ffff12a-8dde-4ae8-8b1e-14e43b435b69 7 2024-08-18T13:39:12.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 Processor -- start 2024-08-18T13:39:12.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- start start 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c159360 0x7f5c7c0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5c7c0ae8a0 0x7f5c7c14f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c0aed00 con 0x7f5c7c0c66e0 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7c0c5560 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c0c4600 con 0x7f5c7c159360 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c81073700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c159360 0x7f5c7c0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82075700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5c7c0ae8a0 0x7f5c7c14f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c81073700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c159360 0x7f5c7c0ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:53384/0 (socket says 172.21.15.178:53384) 2024-08-18T13:39:12.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c81073700 1 -- 172.21.15.178:0/2780707519 learned_addr learned my addr 172.21.15.178:0/2780707519 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82075700 1 -- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c159360 msgr2=0x7f5c7c0ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:12.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82075700 1 --2- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c159360 0x7f5c7c0ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82075700 1 -- 172.21.15.178:0/2780707519 >> v1:172.21.15.118:6789/0 conn(0x7f5c7c0c66e0 legacy=0x7f5c7c0c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82075700 1 -- 172.21.15.178:0/2780707519 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0c48e0 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c81073700 1 --2- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c159360 0x7f5c7c0ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:12.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82075700 1 --2- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c0ae8a0 0x7f5c7c14f1b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009e60 tx=0x7f5c74009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db88bf66ef7c7a73 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c80872700 1 -- 172.21.15.178:0/2780707519 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400e070 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c80872700 1 -- 172.21.15.178:0/2780707519 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7400bb60 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c80872700 1 -- 172.21.15.178:0/2780707519 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74012430 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c0ae8a0 msgr2=0x7f5c7c14f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 --2- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c0ae8a0 0x7f5c7c14f1b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009e60 tx=0x7f5c74009780 comp rx=0 tx=0).stop 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- 172.21.15.178:0/2780707519 shutdown_connections 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 --2- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c0ae8a0 0x7f5c7c14f1b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 --2- 172.21.15.178:0/2780707519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c159360 0x7f5c7c0ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- 172.21.15.178:0/2780707519 >> 172.21.15.178:0/2780707519 conn(0x7f5c7c06d630 msgr2=0x7f5c7c06da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- 172.21.15.178:0/2780707519 shutdown_connections 2024-08-18T13:39:12.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.696+0000 7f5c82876700 1 -- 172.21.15.178:0/2780707519 wait complete. 2024-08-18T13:39:12.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 Processor -- start 2024-08-18T13:39:12.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 -- start start 2024-08-18T13:39:12.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c0ae8a0 0x7f5c7c0c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c7c0c66e0 0x7f5c7c1685b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c159360 0x7f5c7c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c165a50 con 0x7f5c7c0c66e0 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c0695d0 con 0x7f5c7c159360 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c165ee0 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c0ae8a0 0x7f5c7c0c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c0ae8a0 0x7f5c7c0c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:53392/0 (socket says 172.21.15.178:53392) 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 -- 172.21.15.178:0/2771358895 learned_addr learned my addr 172.21.15.178:0/2771358895 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82075700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c159360 0x7f5c7c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81073700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c7c0c66e0 0x7f5c7c1685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 -- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c159360 msgr2=0x7f5c7c14d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c159360 0x7f5c7c14d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 -- 172.21.15.178:0/2771358895 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c7c0c66e0 msgr2=0x7f5c7c1685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c7c0c66e0 0x7f5c7c1685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 -- 172.21.15.178:0/2771358895 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0094f0 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82075700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c159360 0x7f5c7c14d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c81874700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c0ae8a0 0x7f5c7c0c8ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c590 tx=0x7f5c7800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c71ffb700 1 -- 172.21.15.178:0/2771358895 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78011590 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c74009a20 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c71ffb700 1 -- 172.21.15.178:0/2771358895 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c78011710 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c71ffb700 1 -- 172.21.15.178:0/2771358895 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78011ad0 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.700+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c069d70 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.704+0000 7f5c71ffb700 1 -- 172.21.15.178:0/2771358895 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5c7801c440 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.704+0000 7f5c71ffb700 1 -- 172.21.15.178:0/2771358895 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f5c78012a90 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.704+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c064d70 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.708+0000 7f5c71ffb700 1 -- 172.21.15.178:0/2771358895 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5c7c064d70 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.780+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8ffff12a-8dde-4ae8-8b1e-14e43b435b69", "id": 7} v 0) v1 -- 0x7f5c7c071410 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.784+0000 7f5c71ffb700 1 -- 172.21.15.178:0/2771358895 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8ffff12a-8dde-4ae8-8b1e-14e43b435b69", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c78017070 con 0x7f5c7c0ae8a0 2024-08-18T13:39:12.792 INFO:teuthology.orchestra.run.smithi178.stdout:7 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c0ae8a0 msgr2=0x7f5c7c0c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c0ae8a0 0x7f5c7c0c8ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c590 tx=0x7f5c7800c8a0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 shutdown_connections 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c7c159360 0x7f5c7c14d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c7c0c66e0 0x7f5c7c1685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 --2- 172.21.15.178:0/2771358895 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c7c0ae8a0 0x7f5c7c0c8ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 >> 172.21.15.178:0/2771358895 conn(0x7f5c7c06d630 msgr2=0x7f5c7c14e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 shutdown_connections 2024-08-18T13:39:12.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-08-18T13:39:12.788+0000 7f5c82876700 1 -- 172.21.15.178:0/2771358895 wait complete. 2024-08-18T13:39:12.805 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-18T13:39:12.805 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-18T13:39:12.808 INFO:tasks.ceph.osd.0:Started 2024-08-18T13:39:12.808 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-18T13:39:12.808 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-18T13:39:12.810 INFO:tasks.ceph.osd.1:Started 2024-08-18T13:39:12.810 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-18T13:39:12.811 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-18T13:39:12.813 INFO:tasks.ceph.osd.2:Started 2024-08-18T13:39:12.813 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-18T13:39:12.813 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-18T13:39:12.816 INFO:tasks.ceph.osd.3:Started 2024-08-18T13:39:12.816 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-18T13:39:12.816 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-18T13:39:12.818 INFO:tasks.ceph.osd.4:Started 2024-08-18T13:39:12.818 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-18T13:39:12.819 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-18T13:39:12.820 INFO:tasks.ceph.osd.5:Started 2024-08-18T13:39:12.821 INFO:tasks.ceph.osd.6:Restarting daemon 2024-08-18T13:39:12.821 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-08-18T13:39:12.823 INFO:tasks.ceph.osd.6:Started 2024-08-18T13:39:12.823 INFO:tasks.ceph.osd.7:Restarting daemon 2024-08-18T13:39:12.823 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-08-18T13:39:12.825 INFO:tasks.ceph.osd.7:Started 2024-08-18T13:39:12.826 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:12.936 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.931+0000 7ff282586700 1 Processor -- start 2024-08-18T13:39:12.936 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.931+0000 7ff282586700 1 -- start start 2024-08-18T13:39:12.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.931+0000 7ff282586700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff27c06b850 0x7ff27c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.931+0000 7ff282586700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff27c06bc20 0x7ff27c0c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.931+0000 7ff282586700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c14c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.931+0000 7ff282586700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c0bd5b0 con 0x7ff27c06bc20 2024-08-18T13:39:12.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c068b00 con 0x7ff27c06b850 2024-08-18T13:39:12.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c06fa50 con 0x7ff27c0c2b70 2024-08-18T13:39:12.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff27c06bc20 0x7ff27c0c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff280b23700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff27c06b850 0x7ff27c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff27c06bc20 0x7ff27c0c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36408/0 (socket says 172.21.15.118:36408) 2024-08-18T13:39:12.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 -- 172.21.15.118:0/1799189441 learned_addr learned my addr 172.21.15.118:0/1799189441 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 -- 172.21.15.118:0/1799189441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 msgr2=0x7ff27c14c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 --2- 172.21.15.118:0/1799189441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c14c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 -- 172.21.15.118:0/1799189441 >> v2:172.21.15.178:3300/0 conn(0x7ff27c06b850 msgr2=0x7ff27c152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 --2- 172.21.15.118:0/1799189441 >> v2:172.21.15.178:3300/0 conn(0x7ff27c06b850 0x7ff27c152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 -- 172.21.15.118:0/1799189441 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0be950 con 0x7ff27c06bc20 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27bfff700 1 --2- 172.21.15.118:0/1799189441 >> v2:172.21.15.178:3300/0 conn(0x7ff27c06b850 0x7ff27c152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff280b23700 1 --2- 172.21.15.118:0/1799189441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c14c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27b7fe700 1 --2- 172.21.15.118:0/1799189441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c0c2630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0c2270 tx=0x7ff270009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe7d30f1a05bc6d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27affd700 1 -- 172.21.15.118:0/1799189441 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000b640 con 0x7ff27c06bc20 2024-08-18T13:39:12.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27affd700 1 -- 172.21.15.118:0/1799189441 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27000b7a0 con 0x7ff27c06bc20 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff27affd700 1 -- 172.21.15.118:0/1799189441 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000bb40 con 0x7ff27c06bc20 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 -- 172.21.15.118:0/1799189441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 msgr2=0x7ff27c0c2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 --2- 172.21.15.118:0/1799189441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c0c2630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0c2270 tx=0x7ff270009250 comp rx=0 tx=0).stop 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 -- 172.21.15.118:0/1799189441 shutdown_connections 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 --2- 172.21.15.118:0/1799189441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c14c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 --2- 172.21.15.118:0/1799189441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c0c2630 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 --2- 172.21.15.118:0/1799189441 >> v2:172.21.15.178:3300/0 conn(0x7ff27c06b850 0x7ff27c152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 -- 172.21.15.118:0/1799189441 >> 172.21.15.118:0/1799189441 conn(0x7ff27c0c7c40 msgr2=0x7ff27c0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 -- 172.21.15.118:0/1799189441 shutdown_connections 2024-08-18T13:39:12.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.935+0000 7ff282586700 1 -- 172.21.15.118:0/1799189441 wait complete. 2024-08-18T13:39:12.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 Processor -- start 2024-08-18T13:39:12.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 -- start start 2024-08-18T13:39:12.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff27c06b850 0x7ff27c163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c162530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:12.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c163480 con 0x7ff27c06bc20 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c144210 con 0x7ff27c06b850 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff282586700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c166290 con 0x7ff27c0c2b70 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff27b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c14b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff27b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c14b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36410/0 (socket says 172.21.15.118:36410) 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff27b7fe700 1 -- 172.21.15.118:0/3144803655 learned_addr learned my addr 172.21.15.118:0/3144803655 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff27bfff700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff27c06b850 0x7ff27c163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.939+0000 7ff280b23700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff27bfff700 1 -- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 msgr2=0x7ff27c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff27bfff700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff27bfff700 1 -- 172.21.15.118:0/3144803655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 msgr2=0x7ff27c14b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff27bfff700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c14b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff27bfff700 1 -- 172.21.15.118:0/3144803655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274009a20 con 0x7ff27c06b850 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff27bfff700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff27c06b850 0x7ff27c163a10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff274009fd0 tx=0x7ff27400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:12.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff263fff700 1 -- 172.21.15.118:0/3144803655 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400cbd0 con 0x7ff27c06b850 2024-08-18T13:39:12.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff263fff700 1 -- 172.21.15.118:0/3144803655 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400f040 con 0x7ff27c06b850 2024-08-18T13:39:12.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff263fff700 1 -- 172.21.15.118:0/3144803655 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274014480 con 0x7ff27c06b850 2024-08-18T13:39:12.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c009420 con 0x7ff27c06b850 2024-08-18T13:39:12.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2700094f0 con 0x7ff27c06b850 2024-08-18T13:39:12.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff263fff700 1 -- 172.21.15.118:0/3144803655 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff274014b00 con 0x7ff27c06b850 2024-08-18T13:39:12.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.943+0000 7ff263fff700 1 -- 172.21.15.118:0/3144803655 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7ff27401d400 con 0x7ff27c06b850 2024-08-18T13:39:12.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.947+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27c0642f0 con 0x7ff27c06b850 2024-08-18T13:39:12.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:12.951+0000 7ff263fff700 1 -- 172.21.15.118:0/3144803655 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff27403b7a0 con 0x7ff27c06b850 2024-08-18T13:39:12.963 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:12.956+0000 7fdfd8c41000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-18T13:39:12.987 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:12.983+0000 7f3a305dc000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-18T13:39:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.019+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff27c0ad6f0 con 0x7ff27c06b850 2024-08-18T13:39:13.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff263fff700 1 -- 172.21.15.118:0/3144803655 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7ff27402a070 con 0x7ff27c06b850 2024-08-18T13:39:13.026 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:13.027 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":10,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:12.787115+0000","last_up_change":"0.000000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff27c06b850 msgr2=0x7ff27c163a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff27c06b850 0x7ff27c163a10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff274009fd0 tx=0x7ff27400b040 comp rx=0 tx=0).stop 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 shutdown_connections 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff27c0c2b70 0x7ff27c162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff27c06bc20 0x7ff27c14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 --2- 172.21.15.118:0/3144803655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff27c06b850 0x7ff27c163a10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 >> 172.21.15.118:0/3144803655 conn(0x7ff27c0c7c40 msgr2=0x7ff27c0c23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 shutdown_connections 2024-08-18T13:39:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:13.023+0000 7ff282586700 1 -- 172.21.15.118:0/3144803655 wait complete. 2024-08-18T13:39:13.039 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-18T13:39:13.039 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-18T13:39:13.081 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:13.076+0000 7fdfd8c41000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-18T13:39:13.106 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:13.103+0000 7f3a305dc000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-18T13:39:13.213 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-08-18T13:39:13.207+0000 7f9b206e63c0 -1 Falling back to public interface 2024-08-18T13:39:13.213 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-08-18T13:39:13.207+0000 7f90f8a7c3c0 -1 Falling back to public interface 2024-08-18T13:39:13.213 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-08-18T13:39:13.207+0000 7f90f7a45700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-08-18T13:39:13.219 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-08-18T13:39:13.215+0000 7f94af2693c0 -1 Falling back to public interface 2024-08-18T13:39:13.402 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:13.396+0000 7fdfd8c41000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-18T13:39:13.418 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-08-18T13:39:13.412+0000 7f4111bdc3c0 -1 Falling back to public interface 2024-08-18T13:39:13.426 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:13.423+0000 7f3a305dc000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-18T13:39:13.427 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-08-18T13:39:13.423+0000 7f4abc1953c0 -1 Falling back to public interface 2024-08-18T13:39:13.442 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-08-18T13:39:13.436+0000 7f7b6eef83c0 -1 Falling back to public interface 2024-08-18T13:39:13.458 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-08-18T13:39:13.452+0000 7f69433923c0 -1 Falling back to public interface 2024-08-18T13:39:13.458 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-08-18T13:39:13.452+0000 7ff1073093c0 -1 Falling back to public interface 2024-08-18T13:39:13.582 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:13.576+0000 7fdfd8c41000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-18T13:39:13.591 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:13.587+0000 7f3a305dc000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-18T13:39:13.656 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:13.651+0000 7f3a305dc000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-18T13:39:13.708 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:13.703+0000 7f3a305dc000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-18T13:39:13.719 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:13.712+0000 7fdfd8c41000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-18T13:39:13.764 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:13.759+0000 7f3a305dc000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-18T13:39:13.775 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:13.768+0000 7fdfd8c41000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-18T13:39:13.837 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:13.832+0000 7fdfd8c41000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-18T13:39:13.844 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:13.933 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:13.927+0000 7f3a305dc000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-18T13:39:13.937 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.931+0000 7f497d0d9700 1 Processor -- start 2024-08-18T13:39:13.937 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.931+0000 7f497d0d9700 1 -- start start 2024-08-18T13:39:13.939 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f497814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f497806d260 0x7f497814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f497806cdb0 0x7f4978068610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49781563d0 con 0x7f497806d260 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780c6140 con 0x7f497806cdb0 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4978155800 con 0x7f49780bf640 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f497806d260 0x7f497814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f497806d260 0x7f497814c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:38418/0 (socket says 172.21.15.118:38418) 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 -- 172.21.15.118:0/3963400204 learned_addr learned my addr 172.21.15.118:0/3963400204 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 -- 172.21.15.118:0/3963400204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 msgr2=0x7f497814bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f4977fff700 1 --2- 172.21.15.118:0/3963400204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f497814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497c8d8700 1 --2- 172.21.15.118:0/3963400204 >> v2:172.21.15.178:3300/0 conn(0x7f497806cdb0 0x7f4978068610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 --2- 172.21.15.118:0/3963400204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f497814bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.940 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 -- 172.21.15.118:0/3963400204 >> v2:172.21.15.178:3300/0 conn(0x7f497806cdb0 msgr2=0x7f4978068610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 --2- 172.21.15.118:0/3963400204 >> v2:172.21.15.178:3300/0 conn(0x7f497806cdb0 0x7f4978068610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 -- 172.21.15.118:0/3963400204 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978155aa0 con 0x7f497806d260 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f4977fff700 1 --2- 172.21.15.118:0/3963400204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f497814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497c8d8700 1 --2- 172.21.15.118:0/3963400204 >> v2:172.21.15.178:3300/0 conn(0x7f497806cdb0 0x7f4978068610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f496ffff700 1 --2- 172.21.15.118:0/3963400204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806d260 0x7f497814c4f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49780c56a0 tx=0x7f4960009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75f0a3c23b9bade server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f49777fe700 1 -- 172.21.15.118:0/3963400204 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000b860 con 0x7f497806d260 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f49777fe700 1 -- 172.21.15.118:0/3963400204 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496000b9c0 con 0x7f497806d260 2024-08-18T13:39:13.941 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f49777fe700 1 -- 172.21.15.118:0/3963400204 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000bd60 con 0x7f497806d260 2024-08-18T13:39:13.942 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- 172.21.15.118:0/3963400204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806d260 msgr2=0x7f497814c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:13.942 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 --2- 172.21.15.118:0/3963400204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806d260 0x7f497814c4f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49780c56a0 tx=0x7f4960009250 comp rx=0 tx=0).stop 2024-08-18T13:39:13.942 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- 172.21.15.118:0/3963400204 shutdown_connections 2024-08-18T13:39:13.942 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 --2- 172.21.15.118:0/3963400204 >> v2:172.21.15.178:3300/0 conn(0x7f497806cdb0 0x7f4978068610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.942 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 --2- 172.21.15.118:0/3963400204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806d260 0x7f497814c4f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.942 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 --2- 172.21.15.118:0/3963400204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f497814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.942 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- 172.21.15.118:0/3963400204 >> 172.21.15.118:0/3963400204 conn(0x7f49780c96c0 msgr2=0x7f49780c56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:13.943 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- 172.21.15.118:0/3963400204 shutdown_connections 2024-08-18T13:39:13.943 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.935+0000 7f497d0d9700 1 -- 172.21.15.118:0/3963400204 wait complete. 2024-08-18T13:39:13.943 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 Processor -- start 2024-08-18T13:39:13.943 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 -- start start 2024-08-18T13:39:13.946 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806cdb0 0x7f4978151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:13.946 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f497806d260 0x7f4978151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:13.946 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f49781680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:13.946 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978150040 con 0x7f497806cdb0 2024-08-18T13:39:13.946 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978150ea0 con 0x7f497806d260 2024-08-18T13:39:13.946 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497d0d9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49780bfd00 con 0x7f49780bf640 2024-08-18T13:39:13.946 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806cdb0 0x7f4978151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806cdb0 0x7f4978151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:38424/0 (socket says 172.21.15.118:38424) 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 -- 172.21.15.118:0/185623855 learned_addr learned my addr 172.21.15.118:0/185623855 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f497c8d8700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f49781680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f496ffff700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f497806d260 0x7f4978151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 -- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 msgr2=0x7f49781680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f49781680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 -- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f497806d260 msgr2=0x7f4978151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f497806d260 0x7f4978151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f4977fff700 1 -- 172.21.15.118:0/185623855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600094f0 con 0x7f497806cdb0 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.939+0000 7f496ffff700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f497806d260 0x7f4978151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f4977fff700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806cdb0 0x7f4978151fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f496800c860 tx=0x7f4968008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f4974ff9700 1 -- 172.21.15.118:0/185623855 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968011600 con 0x7f497806cdb0 2024-08-18T13:39:13.947 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f4974ff9700 1 -- 172.21.15.118:0/185623855 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968011760 con 0x7f497806cdb0 2024-08-18T13:39:13.948 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4978150290 con 0x7f497806cdb0 2024-08-18T13:39:13.948 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f4974ff9700 1 -- 172.21.15.118:0/185623855 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968011b20 con 0x7f497806cdb0 2024-08-18T13:39:13.948 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4978150850 con 0x7f497806cdb0 2024-08-18T13:39:13.948 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f4974ff9700 1 -- 172.21.15.118:0/185623855 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f496801a450 con 0x7f497806cdb0 2024-08-18T13:39:13.948 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f4974ff9700 1 -- 172.21.15.118:0/185623855 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f496800aad0 con 0x7f497806cdb0 2024-08-18T13:39:13.948 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.943+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4978064b10 con 0x7f497806cdb0 2024-08-18T13:39:13.950 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:13.947+0000 7f4974ff9700 1 -- 172.21.15.118:0/185623855 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f496801f060 con 0x7f497806cdb0 2024-08-18T13:39:14.016 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:14.008+0000 7fdfd8c41000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-18T13:39:14.029 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.023+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f497814fac0 con 0x7f497806cdb0 2024-08-18T13:39:14.030 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f4974ff9700 1 -- 172.21.15.118:0/185623855 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f4968028020 con 0x7f497806cdb0 2024-08-18T13:39:14.030 INFO:teuthology.misc.health.smithi118.stdout: 2024-08-18T13:39:14.030 INFO:teuthology.misc.health.smithi118.stdout:{"epoch":10,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:12.787115+0000","last_up_change":"0.000000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:14.032 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806cdb0 msgr2=0x7f4978151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806cdb0 0x7f4978151fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f496800c860 tx=0x7f4968008040 comp rx=0 tx=0).stop 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 shutdown_connections 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49780bf640 0x7f49781680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f497806d260 0x7f4978151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 --2- 172.21.15.118:0/185623855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f497806cdb0 0x7f4978151fb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 >> 172.21.15.118:0/185623855 conn(0x7f49780c96c0 msgr2=0x7f497814bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 shutdown_connections 2024-08-18T13:39:14.033 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:14.027+0000 7f497d0d9700 1 -- 172.21.15.118:0/185623855 wait complete. 2024-08-18T13:39:14.049 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-08-18T13:39:14.124 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:14.119+0000 7f3a305dc000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-18T13:39:14.189 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:14.183+0000 7f3a305dc000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-18T13:39:14.199 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:14.192+0000 7fdfd8c41000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-18T13:39:14.269 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:14.264+0000 7fdfd8c41000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-18T13:39:14.364 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:14.359+0000 7f3a305dc000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-18T13:39:14.437 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:14.431+0000 7f3a305dc000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-18T13:39:14.444 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:14.436+0000 7fdfd8c41000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-18T13:39:14.511 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:14.504+0000 7fdfd8c41000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-18T13:39:15.159 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:15.155+0000 7f3a305dc000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-18T13:39:15.239 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-08-18T13:39:15.232+0000 7fdfd8c41000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-18T13:39:16.198 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:16.194+0000 7f3a19654700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-18T13:39:16.200 INFO:tasks.ceph.mgr.y.smithi118.stderr:2024-08-18T13:39:16.194+0000 7f3a19654700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-18T13:39:16.859 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-08-18T13:39:16.854+0000 7f90f8a7c3c0 -1 osd.3 0 log_to_monitors true 2024-08-18T13:39:16.973 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-08-18T13:39:16.966+0000 7f4abc1953c0 -1 osd.0 0 log_to_monitors true 2024-08-18T13:39:17.005 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-08-18T13:39:17.002+0000 7f9b206e63c0 -1 osd.2 0 log_to_monitors true 2024-08-18T13:39:17.079 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-08-18T13:39:17.074+0000 7f94af2693c0 -1 osd.1 0 log_to_monitors true 2024-08-18T13:39:17.081 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-08-18T13:39:17.076+0000 7f69433923c0 -1 osd.5 0 log_to_monitors true 2024-08-18T13:39:17.190 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-08-18T13:39:17.184+0000 7f4111bdc3c0 -1 osd.7 0 log_to_monitors true 2024-08-18T13:39:17.299 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-08-18T13:39:17.292+0000 7f7b6eef83c0 -1 osd.4 0 log_to_monitors true 2024-08-18T13:39:17.324 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-08-18T13:39:17.316+0000 7ff1073093c0 -1 osd.6 0 log_to_monitors true 2024-08-18T13:39:19.091 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-08-18T13:39:19.084+0000 7f6940357700 -1 osd.5 0 waiting for initial osdmap 2024-08-18T13:39:19.097 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-08-18T13:39:19.092+0000 7f69378d5700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:19.738 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-08-18T13:39:19.734+0000 7f94ac22e700 -1 osd.1 0 waiting for initial osdmap 2024-08-18T13:39:19.742 INFO:tasks.ceph.osd.1.smithi118.stderr:2024-08-18T13:39:19.738+0000 7f94a37ac700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:19.758 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-08-18T13:39:19.754+0000 7f9b1be86700 -1 osd.2 0 waiting for initial osdmap 2024-08-18T13:39:19.765 INFO:tasks.ceph.osd.2.smithi118.stderr:2024-08-18T13:39:19.758+0000 7f9b14c29700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:19.788 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-08-18T13:39:19.782+0000 7f4ab7935700 -1 osd.0 0 waiting for initial osdmap 2024-08-18T13:39:19.793 INFO:tasks.ceph.osd.0.smithi118.stderr:2024-08-18T13:39:19.786+0000 7f4ab06d8700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:19.799 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-08-18T13:39:19.794+0000 7f90f5a41700 -1 osd.3 0 waiting for initial osdmap 2024-08-18T13:39:19.804 INFO:tasks.ceph.osd.3.smithi118.stderr:2024-08-18T13:39:19.798+0000 7f90ecfbf700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:20.396 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-08-18T13:39:20.387+0000 7f7b6bebd700 -1 osd.4 0 waiting for initial osdmap 2024-08-18T13:39:20.400 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-08-18T13:39:20.395+0000 7f7b6343b700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:20.433 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-08-18T13:39:20.427+0000 7ff1042ce700 -1 osd.6 0 waiting for initial osdmap 2024-08-18T13:39:20.442 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-08-18T13:39:20.435+0000 7ff0fb84c700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:20.455 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-08-18T13:39:20.447+0000 7f410d37c700 -1 osd.7 0 waiting for initial osdmap 2024-08-18T13:39:20.461 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-08-18T13:39:20.455+0000 7f410611f700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T13:39:20.855 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:20.953 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.946+0000 7f823359e700 1 Processor -- start 2024-08-18T13:39:20.954 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823359e700 1 -- start start 2024-08-18T13:39:20.956 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823359e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82340c2d50 0x7f82340c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:20.956 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8234068510 0x7f823406f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:20.956 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823359e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823406f9b0 con 0x7f82340c3940 2024-08-18T13:39:20.956 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823359e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823406ca30 con 0x7f82340c2d50 2024-08-18T13:39:20.956 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82340beb90 con 0x7f8234068510 2024-08-18T13:39:20.957 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823259c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82340c2d50 0x7f82340c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:20.957 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f8229d9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8234068510 0x7f823406f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:20.957 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f8229d9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8234068510 0x7f823406f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:49168/0 (socket says 172.21.15.118:49168) 2024-08-18T13:39:20.957 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f8229d9b700 1 -- 172.21.15.118:0/74248869 learned_addr learned my addr 172.21.15.118:0/74248869 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823259c700 1 -- 172.21.15.118:0/74248869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8234068510 msgr2=0x7f823406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823259c700 1 --2- 172.21.15.118:0/74248869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8234068510 0x7f823406f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823259c700 1 -- 172.21.15.118:0/74248869 >> v1:172.21.15.178:6789/0 conn(0x7f82340c3940 legacy=0x7f82340c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823259c700 1 -- 172.21.15.118:0/74248869 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823406c750 con 0x7f82340c2d50 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f8231d9b700 1 -- 172.21.15.118:0/74248869 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237957305 0 0) 0x7f822c001490 con 0x7f82340c3940 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.950+0000 7f823259c700 1 --2- 172.21.15.118:0/74248869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c2d50 0x7f82340c3120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82240098e0 tx=0x7f82240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf539f6d18dedcb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f8231d9b700 1 -- 172.21.15.118:0/74248869 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822400b6d0 con 0x7f82340c2d50 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f8231d9b700 1 -- 172.21.15.118:0/74248869 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822400b830 con 0x7f82340c2d50 2024-08-18T13:39:20.958 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f8231d9b700 1 -- 172.21.15.118:0/74248869 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822400bbd0 con 0x7f82340c2d50 2024-08-18T13:39:20.959 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 -- 172.21.15.118:0/74248869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c2d50 msgr2=0x7f82340c3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:20.959 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 --2- 172.21.15.118:0/74248869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c2d50 0x7f82340c3120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82240098e0 tx=0x7f82240091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:20.959 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 -- 172.21.15.118:0/74248869 shutdown_connections 2024-08-18T13:39:20.959 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 --2- 172.21.15.118:0/74248869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8234068510 0x7f823406f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:20.959 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 --2- 172.21.15.118:0/74248869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c2d50 0x7f82340c3120 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:20.960 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 -- 172.21.15.118:0/74248869 >> 172.21.15.118:0/74248869 conn(0x7f823406adf0 msgr2=0x7f823406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:20.960 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 -- 172.21.15.118:0/74248869 shutdown_connections 2024-08-18T13:39:20.960 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 -- 172.21.15.118:0/74248869 wait complete. 2024-08-18T13:39:20.960 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 Processor -- start 2024-08-18T13:39:20.960 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.954+0000 7f823359e700 1 -- start start 2024-08-18T13:39:20.962 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8234068510 0x7f82340c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:20.962 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82340c2d50 0x7f823414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:20.962 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c3940 0x7f823414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:20.962 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82340c72a0 con 0x7f8234068510 2024-08-18T13:39:20.962 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82340c8fe0 con 0x7f82340c3940 2024-08-18T13:39:20.962 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82341449c0 con 0x7f82340c2d50 2024-08-18T13:39:20.962 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8234068510 0x7f82340c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:20.963 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8234068510 0x7f82340c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:38536/0 (socket says 172.21.15.118:38536) 2024-08-18T13:39:20.963 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 -- 172.21.15.118:0/1915843114 learned_addr learned my addr 172.21.15.118:0/1915843114 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:20.963 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f8229d9b700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82340c2d50 0x7f823414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:20.963 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f8232d9d700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c3940 0x7f823414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:20.964 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 -- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82340c2d50 msgr2=0x7f823414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:20.964 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82340c2d50 0x7f823414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:20.964 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 -- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c3940 msgr2=0x7f823414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:20.964 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c3940 0x7f823414e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:20.964 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 -- 172.21.15.118:0/1915843114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8224009420 con 0x7f8234068510 2024-08-18T13:39:20.964 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823259c700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8234068510 0x7f82340c8080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f823406b510 tx=0x7f8224017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:20.964 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f822affd700 1 -- 172.21.15.118:0/1915843114 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224020070 con 0x7f8234068510 2024-08-18T13:39:20.965 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f822affd700 1 -- 172.21.15.118:0/1915843114 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822400bbb0 con 0x7f8234068510 2024-08-18T13:39:20.965 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82341419c0 con 0x7f8234068510 2024-08-18T13:39:20.965 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f822affd700 1 -- 172.21.15.118:0/1915843114 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224011670 con 0x7f8234068510 2024-08-18T13:39:20.965 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8234144f20 con 0x7f8234068510 2024-08-18T13:39:20.965 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.958+0000 7f822affd700 1 -- 172.21.15.118:0/1915843114 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f8224011a60 con 0x7f8234068510 2024-08-18T13:39:20.965 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.962+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823414b270 con 0x7f8234068510 2024-08-18T13:39:20.966 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.962+0000 7f822affd700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8218060800 0x7f8218062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:20.967 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.962+0000 7f8229d9b700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8218060800 0x7f8218062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:20.967 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.962+0000 7f822affd700 1 -- 172.21.15.118:0/1915843114 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3627+0+0 (secure 0 0 0) 0x7f822407f510 con 0x7f8234068510 2024-08-18T13:39:20.967 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.962+0000 7f8229d9b700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8218060800 0x7f8218062cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8234149c60 tx=0x7f821c009250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:20.976 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:20.970+0000 7f822affd700 1 -- 172.21.15.118:0/1915843114 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f823414b270 con 0x7f8234068510 2024-08-18T13:39:21.131 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.126+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f823414b270 con 0x7f8234068510 2024-08-18T13:39:21.133 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.126+0000 7f822affd700 1 -- 172.21.15.118:0/1915843114 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9189 (secure 0 0 0) 0x7f823414b270 con 0x7f8234068510 2024-08-18T13:39:21.133 INFO:teuthology.misc.health.smithi118.stdout: 2024-08-18T13:39:21.133 INFO:teuthology.misc.health.smithi118.stdout:{"epoch":14,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:20.236082+0000","last_up_change":"2024-08-18T13:39:20.236082+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:21.135 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8218060800 msgr2=0x7f8218062cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8218060800 0x7f8218062cb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8234149c60 tx=0x7f821c009250 comp rx=0 tx=0).stop 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8234068510 msgr2=0x7f82340c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8234068510 0x7f82340c8080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f823406b510 tx=0x7f8224017000 comp rx=0 tx=0).stop 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 shutdown_connections 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8218060800 0x7f8218062cb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82340c3940 0x7f823414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82340c2d50 0x7f823414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 --2- 172.21.15.118:0/1915843114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8234068510 0x7f82340c8080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:21.136 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 >> 172.21.15.118:0/1915843114 conn(0x7f823406adf0 msgr2=0x7f82341549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:21.137 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 shutdown_connections 2024-08-18T13:39:21.137 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:21.130+0000 7f823359e700 1 -- 172.21.15.118:0/1915843114 wait complete. 2024-08-18T13:39:21.144 DEBUG:teuthology.misc:5 of 8 OSDs are up 2024-08-18T13:39:27.952 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:28.054 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f22309d5700 1 Processor -- start 2024-08-18T13:39:28.055 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f22309d5700 1 -- start start 2024-08-18T13:39:28.055 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f22309d5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f222c0c23d0 0x7f222c159790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.055 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f22309d5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c0c3420 con 0x7f222c0be960 2024-08-18T13:39:28.055 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f22309d5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c0c28a0 con 0x7f222c069820 2024-08-18T13:39:28.055 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f22309d5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c067630 con 0x7f222c0c23d0 2024-08-18T13:39:28.055 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222ad9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f222c0be960 0x7f222c067a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37152/0 (socket says 172.21.15.118:37152) 2024-08-18T13:39:28.056 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222ad9d700 1 -- 172.21.15.118:0/2481897021 learned_addr learned my addr 172.21.15.118:0/2481897021 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:28.058 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f2229d9b700 1 --2- 172.21.15.118:0/2481897021 >> v2:172.21.15.178:3300/0 conn(0x7f222c0c23d0 0x7f222c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.058 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736672249 0 0) 0x7f222c0c3420 con 0x7f222c0be960 2024-08-18T13:39:28.058 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2218003570 con 0x7f222c0be960 2024-08-18T13:39:28.058 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445819297 0 0) 0x7f222c0c28a0 con 0x7f222c069820 2024-08-18T13:39:28.058 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f222c0c3420 con 0x7f222c069820 2024-08-18T13:39:28.058 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4140361280 0 0) 0x7f2218003570 con 0x7f222c0be960 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 >> v2:172.21.15.178:3300/0 conn(0x7f222c0c23d0 msgr2=0x7f222c159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 --2- 172.21.15.118:0/2481897021 >> v2:172.21.15.178:3300/0 conn(0x7f222c0c23d0 0x7f222c159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 >> v1:172.21.15.178:6789/0 conn(0x7f222c069820 legacy=0x7f222c153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0c2b80 con 0x7f222c0be960 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f221c001470 con 0x7f222c0be960 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 >> v1:172.21.15.118:6789/0 conn(0x7f222c0be960 legacy=0x7f222c067a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2218003bf0 0x7f2218003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2218004810 0x7f2218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f221800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c067630 con 0x7f22180051c0 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221800c0b0 con 0x7f2218004810 2024-08-18T13:39:28.059 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221800c270 con 0x7f2218003bf0 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222ad9d700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f221800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222a59c700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2218003bf0 0x7f2218003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f2229d9b700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2218004810 0x7f2218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.050+0000 7f222ad9d700 1 -- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2218003bf0 msgr2=0x7f2218003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222ad9d700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2218003bf0 0x7f2218003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222ad9d700 1 -- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2218004810 msgr2=0x7f2218004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222ad9d700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2218004810 0x7f2218004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222ad9d700 1 -- 172.21.15.118:0/2481897021 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2218003570 con 0x7f22180051c0 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222a59c700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2218003bf0 0x7f2218003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f2229d9b700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2218004810 0x7f2218004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222ad9d700 1 -- 172.21.15.118:0/2481897021 reap_dead start 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222ad9d700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f221800ba60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f222c0bed60 tx=0x7f221c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1121c9ad414081 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.060 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00cb40 con 0x7f22180051c0 2024-08-18T13:39:28.061 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c00cca0 con 0x7f22180051c0 2024-08-18T13:39:28.061 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f222959a700 1 -- 172.21.15.118:0/2481897021 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c005b70 con 0x7f22180051c0 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f22309d5700 1 -- 172.21.15.118:0/2481897021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 msgr2=0x7f221800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f22309d5700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f221800ba60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f222c0bed60 tx=0x7f221c001e10 comp rx=0 tx=0).stop 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f22309d5700 1 -- 172.21.15.118:0/2481897021 shutdown_connections 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f22309d5700 1 --2- 172.21.15.118:0/2481897021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f221800ba60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f22309d5700 1 -- 172.21.15.118:0/2481897021 >> 172.21.15.118:0/2481897021 conn(0x7f222c0c44d0 msgr2=0x7f222c0c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f22309d5700 1 -- 172.21.15.118:0/2481897021 shutdown_connections 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.054+0000 7f22309d5700 1 -- 172.21.15.118:0/2481897021 wait complete. 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 Processor -- start 2024-08-18T13:39:28.062 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 -- start start 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f222c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f222c16c420 0x7f222c0bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f222c0c6d60 0x7f222c0678d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c156020 con 0x7f22180051c0 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c1540b0 con 0x7f222c16c420 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c156230 con 0x7f222c0c6d60 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f222c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f222c154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53342/0 (socket says 172.21.15.118:53342) 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 -- 172.21.15.118:0/1495480782 learned_addr learned my addr 172.21.15.118:0/1495480782 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f2229d9b700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f222c16c420 0x7f222c0bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222ad9d700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f222c0c6d60 0x7f222c0678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.064 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 -- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f222c0c6d60 msgr2=0x7f222c0678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.065 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f222c0c6d60 0x7f222c0678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.065 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 -- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f222c16c420 msgr2=0x7f222c0bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.065 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f222c16c420 0x7f222c0bfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.065 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 -- 172.21.15.118:0/1495480782 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c156510 con 0x7f22180051c0 2024-08-18T13:39:28.065 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f222a59c700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f222c154540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f222400eb00 tx=0x7f2224009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.066 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22227fc700 1 -- 172.21.15.118:0/1495480782 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224006ba0 con 0x7f22180051c0 2024-08-18T13:39:28.066 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22227fc700 1 -- 172.21.15.118:0/1495480782 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224010040 con 0x7f22180051c0 2024-08-18T13:39:28.066 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c1567f0 con 0x7f22180051c0 2024-08-18T13:39:28.066 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22227fc700 1 -- 172.21.15.118:0/1495480782 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240032e0 con 0x7f22180051c0 2024-08-18T13:39:28.066 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.058+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c153d40 con 0x7f22180051c0 2024-08-18T13:39:28.066 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.062+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c064b10 con 0x7f22180051c0 2024-08-18T13:39:28.066 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.062+0000 7f22227fc700 1 -- 172.21.15.118:0/1495480782 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f22240016e0 con 0x7f22180051c0 2024-08-18T13:39:28.067 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.062+0000 7f22227fc700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f220c060860 0x7f220c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.067 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.062+0000 7f22227fc700 1 -- 172.21.15.118:0/1495480782 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f222407c390 con 0x7f22180051c0 2024-08-18T13:39:28.067 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.062+0000 7f2229d9b700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f220c060860 0x7f220c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.068 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.062+0000 7f2229d9b700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f220c060860 0x7f220c062d10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f222c065840 tx=0x7f2214008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.072 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.066+0000 7f22227fc700 1 -- 172.21.15.118:0/1495480782 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f222404cbf0 con 0x7f22180051c0 2024-08-18T13:39:28.238 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.234+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f222c154a80 con 0x7f22180051c0 2024-08-18T13:39:28.240 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.234+0000 7f22227fc700 1 -- 172.21.15.118:0/1495480782 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f2224050210 con 0x7f22180051c0 2024-08-18T13:39:28.240 INFO:teuthology.misc.health.smithi118.stdout: 2024-08-18T13:39:28.240 INFO:teuthology.misc.health.smithi118.stdout:{"epoch":17,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:23.767627+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f220c060860 msgr2=0x7f220c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f220c060860 0x7f220c062d10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f222c065840 tx=0x7f2214008040 comp rx=0 tx=0).stop 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 msgr2=0x7f222c154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f222c154540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f222400eb00 tx=0x7f2224009040 comp rx=0 tx=0).stop 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 shutdown_connections 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f220c060860 0x7f220c062d10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f222c0c6d60 0x7f222c0678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f222c16c420 0x7f222c0bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 --2- 172.21.15.118:0/1495480782 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22180051c0 0x7f222c154540 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.243 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 >> 172.21.15.118:0/1495480782 conn(0x7f222c0c44d0 msgr2=0x7f222c0c57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:28.244 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 shutdown_connections 2024-08-18T13:39:28.244 INFO:teuthology.misc.health.smithi118.stderr:2024-08-18T13:39:28.238+0000 7f22309d5700 1 -- 172.21.15.118:0/1495480782 wait complete. 2024-08-18T13:39:28.253 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-08-18T13:39:28.253 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-18T13:39:28.253 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-18T13:39:28.254 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-18T13:39:28.255 INFO:tasks.ceph.mds.a:Started 2024-08-18T13:39:28.256 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-18T13:39:28.256 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-18T13:39:28.257 INFO:tasks.ceph.mds.c:Started 2024-08-18T13:39:28.257 INFO:tasks.ceph.mds.e:Restarting daemon 2024-08-18T13:39:28.258 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-08-18T13:39:28.259 INFO:tasks.ceph.mds.e:Started 2024-08-18T13:39:28.259 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-18T13:39:28.259 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-18T13:39:28.262 INFO:tasks.ceph.mds.b:Started 2024-08-18T13:39:28.262 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-18T13:39:28.262 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-18T13:39:28.264 INFO:tasks.ceph.mds.d:Started 2024-08-18T13:39:28.264 INFO:tasks.ceph.mds.f:Restarting daemon 2024-08-18T13:39:28.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-08-18T13:39:28.267 INFO:tasks.ceph.mds.f:Started 2024-08-18T13:39:28.267 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-08-18T13:39:28.267 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:28.318 INFO:tasks.ceph.mds.c.smithi118.stdout:starting mds.c at 2024-08-18T13:39:28.326 INFO:tasks.ceph.mds.a.smithi118.stdout:starting mds.a at 2024-08-18T13:39:28.334 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-08-18T13:39:28.335 INFO:tasks.ceph.mds.f.smithi178.stdout:starting mds.f at 2024-08-18T13:39:28.337 INFO:tasks.ceph.mds.e.smithi118.stdout:starting mds.e at 2024-08-18T13:39:28.339 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-08-18T13:39:28.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.358+0000 7fc526974700 1 Processor -- start 2024-08-18T13:39:28.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.358+0000 7fc526974700 1 -- start start 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc5200c2150 0x7fc52005e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc520159120 0x7fc520067d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc52006bca0 0x7fc52006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5200c41f0 con 0x7fc5200c2150 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc520155270 con 0x7fc52006bca0 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52006b240 con 0x7fc520159120 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc5200c2150 0x7fc52005e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc5200c2150 0x7fc52005e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53394/0 (socket says 172.21.15.118:53394) 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 -- 172.21.15.118:0/1251920643 learned_addr learned my addr 172.21.15.118:0/1251920643 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:28.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525171700 1 --2- 172.21.15.118:0/1251920643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc520159120 0x7fc520067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526173700 1 --2- 172.21.15.118:0/1251920643 >> v2:172.21.15.178:3300/0 conn(0x7fc52006bca0 0x7fc52006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 -- 172.21.15.118:0/1251920643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc520159120 msgr2=0x7fc520067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 --2- 172.21.15.118:0/1251920643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc520159120 0x7fc520067d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 -- 172.21.15.118:0/1251920643 >> v2:172.21.15.178:3300/0 conn(0x7fc52006bca0 msgr2=0x7fc52006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 --2- 172.21.15.118:0/1251920643 >> v2:172.21.15.178:3300/0 conn(0x7fc52006bca0 0x7fc52006c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 -- 172.21.15.118:0/1251920643 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5201554e0 con 0x7fc5200c2150 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc525972700 1 --2- 172.21.15.118:0/1251920643 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc5200c2150 0x7fc52005e8d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc52006fff0 tx=0x7fc51c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3f3372ed9c2899 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc524970700 1 -- 172.21.15.118:0/1251920643 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c009e70 con 0x7fc5200c2150 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc524970700 1 -- 172.21.15.118:0/1251920643 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00b860 con 0x7fc5200c2150 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc524970700 1 -- 172.21.15.118:0/1251920643 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00bc00 con 0x7fc5200c2150 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- 172.21.15.118:0/1251920643 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc5200c2150 msgr2=0x7fc52005e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 --2- 172.21.15.118:0/1251920643 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc5200c2150 0x7fc52005e8d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc52006fff0 tx=0x7fc51c009250 comp rx=0 tx=0).stop 2024-08-18T13:39:28.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- 172.21.15.118:0/1251920643 shutdown_connections 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 --2- 172.21.15.118:0/1251920643 >> v2:172.21.15.178:3300/0 conn(0x7fc52006bca0 0x7fc52006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 --2- 172.21.15.118:0/1251920643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc520159120 0x7fc520067d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 --2- 172.21.15.118:0/1251920643 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc5200c2150 0x7fc52005e8d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- 172.21.15.118:0/1251920643 >> 172.21.15.118:0/1251920643 conn(0x7fc5200c81e0 msgr2=0x7fc5200c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- 172.21.15.118:0/1251920643 shutdown_connections 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.362+0000 7fc526974700 1 -- 172.21.15.118:0/1251920643 wait complete. 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 Processor -- start 2024-08-18T13:39:28.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 -- start start 2024-08-18T13:39:28.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc52006bca0 0x7fc5200c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5200c2150 0x7fc520165290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc520159120 0x7fc520157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc520169120 con 0x7fc52006bca0 2024-08-18T13:39:28.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc520158650 con 0x7fc520159120 2024-08-18T13:39:28.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc520168810 con 0x7fc5200c2150 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc52006bca0 0x7fc5200c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc52006bca0 0x7fc5200c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53406/0 (socket says 172.21.15.118:53406) 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 -- 172.21.15.118:0/2182230518 learned_addr learned my addr 172.21.15.118:0/2182230518 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525171700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5200c2150 0x7fc520165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526173700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc520159120 0x7fc520157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 -- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5200c2150 msgr2=0x7fc520165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5200c2150 0x7fc520165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 -- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc520159120 msgr2=0x7fc520157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc520159120 0x7fc520157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 -- 172.21.15.118:0/2182230518 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c0094f0 con 0x7fc52006bca0 2024-08-18T13:39:28.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc525972700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc52006bca0 0x7fc5200c7390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc51c006fd0 tx=0x7fc51c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc515ffb700 1 -- 172.21.15.118:0/2182230518 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00bbe0 con 0x7fc52006bca0 2024-08-18T13:39:28.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc515ffb700 1 -- 172.21.15.118:0/2182230518 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c000c90 con 0x7fc52006bca0 2024-08-18T13:39:28.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5201588c0 con 0x7fc52006bca0 2024-08-18T13:39:28.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc515ffb700 1 -- 172.21.15.118:0/2182230518 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c011630 con 0x7fc52006bca0 2024-08-18T13:39:28.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.366+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc520168db0 con 0x7fc52006bca0 2024-08-18T13:39:28.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.370+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc520152f80 con 0x7fc52006bca0 2024-08-18T13:39:28.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.370+0000 7fc515ffb700 1 -- 172.21.15.118:0/2182230518 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc51c011a60 con 0x7fc52006bca0 2024-08-18T13:39:28.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.370+0000 7fc515ffb700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc504060860 0x7fc504062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.370+0000 7fc515ffb700 1 -- 172.21.15.118:0/2182230518 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc51c07e0d0 con 0x7fc52006bca0 2024-08-18T13:39:28.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.370+0000 7fc525171700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc504060860 0x7fc504062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.370+0000 7fc525171700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc504060860 0x7fc504062d10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc52014ee50 tx=0x7fc5100091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.378+0000 7fc515ffb700 1 -- 172.21.15.118:0/2182230518 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc51c04e7d0 con 0x7fc52006bca0 2024-08-18T13:39:28.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.546+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5200713a0 con 0x7fc52006bca0 2024-08-18T13:39:28.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.546+0000 7fc515ffb700 1 -- 172.21.15.118:0/2182230518 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fc51c051df0 con 0x7fc52006bca0 2024-08-18T13:39:28.552 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:28.552 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":17,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:23.767627+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:28.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc504060860 msgr2=0x7fc504062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc504060860 0x7fc504062d10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc52014ee50 tx=0x7fc5100091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc52006bca0 msgr2=0x7fc5200c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc52006bca0 0x7fc5200c7390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc51c006fd0 tx=0x7fc51c017000 comp rx=0 tx=0).stop 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 shutdown_connections 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc504060860 0x7fc504062d10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc520159120 0x7fc520157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5200c2150 0x7fc520165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 --2- 172.21.15.118:0/2182230518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc52006bca0 0x7fc5200c7390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 >> 172.21.15.118:0/2182230518 conn(0x7fc5200c81e0 msgr2=0x7fc5200c85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 shutdown_connections 2024-08-18T13:39:28.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.550+0000 7fc526974700 1 -- 172.21.15.118:0/2182230518 wait complete. 2024-08-18T13:39:28.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T13:39:21.246221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-18T13:39:28.566 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T13:39:28.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.670+0000 7f3d6b23b700 1 Processor -- start 2024-08-18T13:39:28.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.670+0000 7f3d6b23b700 1 -- start start 2024-08-18T13:39:28.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3d64155710 0x7f3d6406bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155ae0 0x7f3d64155f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d64067a60 con 0x7f3d640c13c0 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6406a770 con 0x7f3d64155710 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c0f30 con 0x7f3d64155ae0 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6aa3a700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f3d640c13c0 0x7f3d6414bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37180/0 (socket says 172.21.15.118:37180) 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6aa3a700 1 -- 172.21.15.118:0/3859255974 learned_addr learned my addr 172.21.15.118:0/3859255974 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69a38700 1 --2- 172.21.15.118:0/3859255974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155ae0 0x7f3d64155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6a239700 1 --2- 172.21.15.118:0/3859255974 >> v2:172.21.15.178:3300/0 conn(0x7f3d64155710 0x7f3d6406bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69237700 1 -- 172.21.15.118:0/3859255974 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381367004 0 0) 0x7f3d64067a60 con 0x7f3d640c13c0 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69237700 1 -- 172.21.15.118:0/3859255974 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d50003570 con 0x7f3d640c13c0 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69a38700 1 -- 172.21.15.118:0/3859255974 >> v2:172.21.15.178:3300/0 conn(0x7f3d64155710 msgr2=0x7f3d6406bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69a38700 1 --2- 172.21.15.118:0/3859255974 >> v2:172.21.15.178:3300/0 conn(0x7f3d64155710 0x7f3d6406bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69a38700 1 -- 172.21.15.118:0/3859255974 >> v1:172.21.15.118:6789/0 conn(0x7f3d640c13c0 legacy=0x7f3d6414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69a38700 1 -- 172.21.15.118:0/3859255974 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64067780 con 0x7f3d64155ae0 2024-08-18T13:39:28.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69a38700 1 --2- 172.21.15.118:0/3859255974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155ae0 0x7f3d64155f10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3d540098e0 tx=0x7f3d540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bfcdd3d7c4ecaa3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69237700 1 -- 172.21.15.118:0/3859255974 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54011630 con 0x7f3d64155ae0 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69237700 1 -- 172.21.15.118:0/3859255974 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d54011790 con 0x7f3d64155ae0 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d69237700 1 -- 172.21.15.118:0/3859255974 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54011b10 con 0x7f3d64155ae0 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- 172.21.15.118:0/3859255974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155ae0 msgr2=0x7f3d64155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/3859255974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155ae0 0x7f3d64155f10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3d540098e0 tx=0x7f3d540091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- 172.21.15.118:0/3859255974 shutdown_connections 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/3859255974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155ae0 0x7f3d64155f10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/3859255974 >> v2:172.21.15.178:3300/0 conn(0x7f3d64155710 0x7f3d6406bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- 172.21.15.118:0/3859255974 >> 172.21.15.118:0/3859255974 conn(0x7f3d6406b1f0 msgr2=0x7f3d6406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- 172.21.15.118:0/3859255974 shutdown_connections 2024-08-18T13:39:28.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.674+0000 7f3d6b23b700 1 -- 172.21.15.118:0/3859255974 wait complete. 2024-08-18T13:39:28.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.678+0000 7f3d6b23b700 1 Processor -- start 2024-08-18T13:39:28.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.678+0000 7f3d6b23b700 1 -- start start 2024-08-18T13:39:28.685 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.678+0000 7f3d6b23b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d640c13c0 0x7f3d641626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.685 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.678+0000 7f3d6b23b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155710 0x7f3d640c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.678+0000 7f3d6b23b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d64155ae0 0x7f3d6414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.678+0000 7f3d6b23b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6414dc40 con 0x7f3d64155ae0 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.678+0000 7f3d6b23b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d64162c00 con 0x7f3d640c13c0 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6b23b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c3580 con 0x7f3d64155710 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6aa3a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d64155ae0 0x7f3d6414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d69a38700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155710 0x7f3d640c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6a239700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d640c13c0 0x7f3d641626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6aa3a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d64155ae0 0x7f3d6414a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53416/0 (socket says 172.21.15.118:53416) 2024-08-18T13:39:28.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6aa3a700 1 -- 172.21.15.118:0/715749739 learned_addr learned my addr 172.21.15.118:0/715749739 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:28.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6a239700 1 -- 172.21.15.118:0/715749739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155710 msgr2=0x7f3d640c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6a239700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155710 0x7f3d640c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6a239700 1 -- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d64155ae0 msgr2=0x7f3d6414a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6a239700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d64155ae0 0x7f3d6414a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6a239700 1 -- 172.21.15.118:0/715749739 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54009420 con 0x7f3d640c13c0 2024-08-18T13:39:28.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6a239700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d640c13c0 0x7f3d641626c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000ea50 tx=0x7f3d600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d5a7fc700 1 -- 172.21.15.118:0/715749739 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000ee90 con 0x7f3d640c13c0 2024-08-18T13:39:28.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d5a7fc700 1 -- 172.21.15.118:0/715749739 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6000c880 con 0x7f3d640c13c0 2024-08-18T13:39:28.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d640c3860 con 0x7f3d640c13c0 2024-08-18T13:39:28.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d5a7fc700 1 -- 172.21.15.118:0/715749739 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000cc20 con 0x7f3d640c13c0 2024-08-18T13:39:28.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d640c2110 con 0x7f3d640c13c0 2024-08-18T13:39:28.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.682+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d640c1f70 con 0x7f3d640c13c0 2024-08-18T13:39:28.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.686+0000 7f3d5a7fc700 1 -- 172.21.15.118:0/715749739 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3d60013470 con 0x7f3d640c13c0 2024-08-18T13:39:28.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.686+0000 7f3d5a7fc700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3d48060860 0x7f3d48062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.686+0000 7f3d5a7fc700 1 -- 172.21.15.118:0/715749739 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3d6007d080 con 0x7f3d640c13c0 2024-08-18T13:39:28.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.686+0000 7f3d69a38700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3d48060860 0x7f3d48062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.686+0000 7f3d69a38700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3d48060860 0x7f3d48062d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d540091e0 tx=0x7f3d54017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.701 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.698+0000 7f3d5a7fc700 1 -- 172.21.15.118:0/715749739 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3d60051330 con 0x7f3d640c13c0 2024-08-18T13:39:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.858+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d640c1f70 con 0x7f3d640c13c0 2024-08-18T13:39:28.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.858+0000 7f3d5a7fc700 1 -- 172.21.15.118:0/715749739 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d60017020 con 0x7f3d640c13c0 2024-08-18T13:39:28.865 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-08-18T13:39:28.867 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3d48060860 msgr2=0x7f3d48062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.867 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3d48060860 0x7f3d48062d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d540091e0 tx=0x7f3d54017000 comp rx=0 tx=0).stop 2024-08-18T13:39:28.867 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d640c13c0 msgr2=0x7f3d641626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.867 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d640c13c0 0x7f3d641626c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000ea50 tx=0x7f3d600091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.867 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 shutdown_connections 2024-08-18T13:39:28.867 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3d48060860 0x7f3d48062d10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d64155ae0 0x7f3d6414a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d64155710 0x7f3d640c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 --2- 172.21.15.118:0/715749739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d640c13c0 0x7f3d641626c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 >> 172.21.15.118:0/715749739 conn(0x7f3d6406b1f0 msgr2=0x7f3d640c0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:28.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 shutdown_connections 2024-08-18T13:39:28.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.862+0000 7f3d6b23b700 1 -- 172.21.15.118:0/715749739 wait complete. 2024-08-18T13:39:28.879 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:39:28.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.982+0000 7f6f1a72c700 1 Processor -- start 2024-08-18T13:39:28.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.982+0000 7f6f1a72c700 1 -- start start 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6f140add40 0x7f6f14155b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f140ae970 0x7f6f1415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f14156060 con 0x7f6f140b15f0 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f14155110 con 0x7f6f140ae970 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1406f340 con 0x7f6f140add40 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f140ae970 0x7f6f1415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6f140ae970 0x7f6f1415fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53426/0 (socket says 172.21.15.118:53426) 2024-08-18T13:39:28.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 -- 172.21.15.118:0/987090457 learned_addr learned my addr 172.21.15.118:0/987090457 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1972a700 1 --2- 172.21.15.118:0/987090457 >> v2:172.21.15.178:3300/0 conn(0x7f6f140add40 0x7f6f14155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 -- 172.21.15.118:0/987090457 >> v2:172.21.15.178:3300/0 conn(0x7f6f140add40 msgr2=0x7f6f14155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 --2- 172.21.15.118:0/987090457 >> v2:172.21.15.178:3300/0 conn(0x7f6f140add40 0x7f6f14155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 -- 172.21.15.118:0/987090457 >> v1:172.21.15.178:6789/0 conn(0x7f6f140b15f0 legacy=0x7f6f140c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 -- 172.21.15.118:0/987090457 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140aeeb0 con 0x7f6f140ae970 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f19f2b700 1 --2- 172.21.15.118:0/987090457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140ae970 0x7f6f1415fab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f140717d0 tx=0x7f6f0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ef5ef022c6f6de server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f0bfff700 1 -- 172.21.15.118:0/987090457 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00b860 con 0x7f6f140ae970 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f0bfff700 1 -- 172.21.15.118:0/987090457 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c00b9c0 con 0x7f6f140ae970 2024-08-18T13:39:28.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f0bfff700 1 -- 172.21.15.118:0/987090457 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00bd60 con 0x7f6f140ae970 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- 172.21.15.118:0/987090457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140ae970 msgr2=0x7f6f1415fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/987090457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140ae970 0x7f6f1415fab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f140717d0 tx=0x7f6f0c009250 comp rx=0 tx=0).stop 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- 172.21.15.118:0/987090457 shutdown_connections 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/987090457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140ae970 0x7f6f1415fab0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/987090457 >> v2:172.21.15.178:3300/0 conn(0x7f6f140add40 0x7f6f14155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- 172.21.15.118:0/987090457 >> 172.21.15.118:0/987090457 conn(0x7f6f140cc600 msgr2=0x7f6f140c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- 172.21.15.118:0/987090457 shutdown_connections 2024-08-18T13:39:28.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.986+0000 7f6f1a72c700 1 -- 172.21.15.118:0/987090457 wait complete. 2024-08-18T13:39:28.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 Processor -- start 2024-08-18T13:39:28.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 -- start start 2024-08-18T13:39:28.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f140add40 0x7f6f140690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f140ae970 0x7f6f140c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140b15f0 0x7f6f140c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f141527d0 con 0x7f6f140b15f0 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140bf680 con 0x7f6f140add40 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14151840 con 0x7f6f140ae970 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140b15f0 0x7f6f140c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140b15f0 0x7f6f140c1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53430/0 (socket says 172.21.15.118:53430) 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 -- 172.21.15.118:0/2551144931 learned_addr learned my addr 172.21.15.118:0/2551144931 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1972a700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f140add40 0x7f6f140690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f18f29700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f140ae970 0x7f6f140c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 -- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f140ae970 msgr2=0x7f6f140c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f140ae970 0x7f6f140c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 -- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f140add40 msgr2=0x7f6f140690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:28.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f140add40 0x7f6f140690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 -- 172.21.15.118:0/2551144931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c0094f0 con 0x7f6f140b15f0 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f18f29700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f140ae970 0x7f6f140c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f19f2b700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140b15f0 0x7f6f140c1b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009290 tx=0x7f6f0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f097fa700 1 -- 172.21.15.118:0/2551144931 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c01a630 con 0x7f6f140b15f0 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f097fa700 1 -- 172.21.15.118:0/2551144931 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c01a790 con 0x7f6f140b15f0 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140bf920 con 0x7f6f140b15f0 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f097fa700 1 -- 172.21.15.118:0/2551144931 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c01ab50 con 0x7f6f140b15f0 2024-08-18T13:39:28.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.990+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f14153210 con 0x7f6f140b15f0 2024-08-18T13:39:28.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.994+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f140713a0 con 0x7f6f140b15f0 2024-08-18T13:39:28.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.994+0000 7f6f097fa700 1 -- 172.21.15.118:0/2551144931 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6f0c0233e0 con 0x7f6f140b15f0 2024-08-18T13:39:28.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.994+0000 7f6f097fa700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ef8060860 0x7f6ef8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:28.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.994+0000 7f6f097fa700 1 -- 172.21.15.118:0/2551144931 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6f0c07e430 con 0x7f6f140b15f0 2024-08-18T13:39:28.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.994+0000 7f6f1972a700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ef8060860 0x7f6ef8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:29.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:28.994+0000 7f6f1972a700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ef8060860 0x7f6ef8062d10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f04006fd0 tx=0x7f6f04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:29.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.002+0000 7f6f097fa700 1 -- 172.21.15.118:0/2551144931 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6f0c04ebd0 con 0x7f6f140b15f0 2024-08-18T13:39:29.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.198+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f140cca00 con 0x7f6f140b15f0 2024-08-18T13:39:29.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.198+0000 7f6f097fa700 1 -- 172.21.15.118:0/2551144931 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7f6f0c015500 con 0x7f6f140b15f0 2024-08-18T13:39:29.204 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:29.204 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-08-18T13:39:29.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ef8060860 msgr2=0x7f6ef8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ef8060860 0x7f6ef8062d10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f04006fd0 tx=0x7f6f04008040 comp rx=0 tx=0).stop 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140b15f0 msgr2=0x7f6f140c1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140b15f0 0x7f6f140c1b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009290 tx=0x7f6f0c017000 comp rx=0 tx=0).stop 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 shutdown_connections 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ef8060860 0x7f6ef8062d10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6f140b15f0 0x7f6f140c1b80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6f140ae970 0x7f6f140c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 --2- 172.21.15.118:0/2551144931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f140add40 0x7f6f140690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 >> 172.21.15.118:0/2551144931 conn(0x7f6f140cc600 msgr2=0x7f6f14159870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 shutdown_connections 2024-08-18T13:39:29.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:29.202+0000 7f6f1a72c700 1 -- 172.21.15.118:0/2551144931 wait complete. 2024-08-18T13:39:29.209 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 2 2024-08-18T13:39:31.219 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:39:31.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd453d5e700 1 Processor -- start 2024-08-18T13:39:31.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd453d5e700 1 -- start start 2024-08-18T13:39:31.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd453d5e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0be710 0x7fd44c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd453d5e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c0c3020 con 0x7fd44c0be340 2024-08-18T13:39:31.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd453d5e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c0c38c0 con 0x7fd44c0c3d50 2024-08-18T13:39:31.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd453d5e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c06e3b0 con 0x7fd44c0be710 2024-08-18T13:39:31.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd451afa700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fd44c0be340 0x7fd44c15a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37194/0 (socket says 172.21.15.118:37194) 2024-08-18T13:39:31.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd451afa700 1 -- 172.21.15.118:0/1191339674 learned_addr learned my addr 172.21.15.118:0/1191339674 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:31.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.330+0000 7fd4512f9700 1 --2- 172.21.15.118:0/1191339674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0be710 0x7fd44c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398128629 0 0) 0x7fd44c0c3020 con 0x7fd44c0be340 2024-08-18T13:39:31.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd438003570 con 0x7fd44c0be340 2024-08-18T13:39:31.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525426656 0 0) 0x7fd44c0c38c0 con 0x7fd44c0c3d50 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd44c0c3020 con 0x7fd44c0c3d50 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd4512f9700 1 -- 172.21.15.118:0/1191339674 >> v1:172.21.15.178:6789/0 conn(0x7fd44c0c3d50 legacy=0x7fd44c06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd4512f9700 1 -- 172.21.15.118:0/1191339674 >> v1:172.21.15.118:6789/0 conn(0x7fd44c0be340 legacy=0x7fd44c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591037601 0 0) 0x7fd438003570 con 0x7fd44c0be340 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd4512f9700 1 -- 172.21.15.118:0/1191339674 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0673d0 con 0x7fd44c0be710 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd4512f9700 1 --2- 172.21.15.118:0/1191339674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0be710 0x7fd44c06db20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd440009910 tx=0x7fd4400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90dd532da18cb3ca server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448001470 con 0x7fd44c0be710 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd440011420 con 0x7fd44c0be710 2024-08-18T13:39:31.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd450af8700 1 -- 172.21.15.118:0/1191339674 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400117c0 con 0x7fd44c0be710 2024-08-18T13:39:31.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd453d5e700 1 -- 172.21.15.118:0/1191339674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0be710 msgr2=0x7fd44c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd453d5e700 1 --2- 172.21.15.118:0/1191339674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0be710 0x7fd44c06db20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd440009910 tx=0x7fd4400091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd453d5e700 1 -- 172.21.15.118:0/1191339674 shutdown_connections 2024-08-18T13:39:31.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd453d5e700 1 --2- 172.21.15.118:0/1191339674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0be710 0x7fd44c06db20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd453d5e700 1 -- 172.21.15.118:0/1191339674 >> 172.21.15.118:0/1191339674 conn(0x7fd44c0c7c40 msgr2=0x7fd44c0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:31.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd453d5e700 1 -- 172.21.15.118:0/1191339674 shutdown_connections 2024-08-18T13:39:31.342 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.334+0000 7fd453d5e700 1 -- 172.21.15.118:0/1191339674 wait complete. 2024-08-18T13:39:31.343 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 Processor -- start 2024-08-18T13:39:31.343 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 -- start start 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd44c0be340 0x7fd44c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd44c0be710 0x7fd44c0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0c3d50 0x7fd44c16aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c15f470 con 0x7fd44c0be710 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0bf3f0 con 0x7fd44c0be340 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd453d5e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c15fe40 con 0x7fd44c0c3d50 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd4512f9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd44c0be710 0x7fd44c0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd4512f9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd44c0be710 0x7fd44c0c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53432/0 (socket says 172.21.15.118:53432) 2024-08-18T13:39:31.346 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd4512f9700 1 -- 172.21.15.118:0/2039422020 learned_addr learned my addr 172.21.15.118:0/2039422020 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:31.346 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd4522fb700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0c3d50 0x7fd44c16aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.346 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.338+0000 7fd451afa700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd44c0be340 0x7fd44c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd4512f9700 1 -- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0c3d50 msgr2=0x7fd44c16aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd4512f9700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0c3d50 0x7fd44c16aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd4512f9700 1 -- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd44c0be340 msgr2=0x7fd44c154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd4512f9700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd44c0be340 0x7fd44c154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd4512f9700 1 -- 172.21.15.118:0/2039422020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440009420 con 0x7fd44c0be710 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd4512f9700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd44c0be710 0x7fd44c0c6b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4400091e0 tx=0x7fd440016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd43dffb700 1 -- 172.21.15.118:0/2039422020 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44001caf0 con 0x7fd44c0be710 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd43dffb700 1 -- 172.21.15.118:0/2039422020 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44001cc50 con 0x7fd44c0be710 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c160100 con 0x7fd44c0be710 2024-08-18T13:39:31.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd43dffb700 1 -- 172.21.15.118:0/2039422020 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44001a450 con 0x7fd44c0be710 2024-08-18T13:39:31.348 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c15f100 con 0x7fd44c0be710 2024-08-18T13:39:31.348 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c0642f0 con 0x7fd44c0be710 2024-08-18T13:39:31.348 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd43dffb700 1 -- 172.21.15.118:0/2039422020 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd44001a840 con 0x7fd44c0be710 2024-08-18T13:39:31.349 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.342+0000 7fd43dffb700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd430060860 0x7fd430062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.349 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.346+0000 7fd451afa700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd430060860 0x7fd430062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.349 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.346+0000 7fd43dffb700 1 -- 172.21.15.118:0/2039422020 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd44007e1a0 con 0x7fd44c0be710 2024-08-18T13:39:31.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.346+0000 7fd451afa700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd430060860 0x7fd430062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4480011e0 tx=0x7fd448000bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:31.357 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.350+0000 7fd43dffb700 1 -- 172.21.15.118:0/2039422020 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd44004e940 con 0x7fd44c0be710 2024-08-18T13:39:31.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.554+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd44c16ac20 con 0x7fd44c0be710 2024-08-18T13:39:31.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.554+0000 7fd43dffb700 1 -- 172.21.15.118:0/2039422020 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5054 (secure 0 0 0) 0x7fd440051f60 con 0x7fd44c0be710 2024-08-18T13:39:31.560 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:31.560 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd430060860 msgr2=0x7fd430062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd430060860 0x7fd430062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4480011e0 tx=0x7fd448000bc0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd44c0be710 msgr2=0x7fd44c0c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd44c0be710 0x7fd44c0c6b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4400091e0 tx=0x7fd440016000 comp rx=0 tx=0).stop 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 shutdown_connections 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd430060860 0x7fd430062d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44c0c3d50 0x7fd44c16aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd44c0be710 0x7fd44c0c6b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 --2- 172.21.15.118:0/2039422020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd44c0be340 0x7fd44c154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 >> 172.21.15.118:0/2039422020 conn(0x7fd44c0c7c40 msgr2=0x7fd44c152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:31.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 shutdown_connections 2024-08-18T13:39:31.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.558+0000 7fd453d5e700 1 -- 172.21.15.118:0/2039422020 wait complete. 2024-08-18T13:39:31.564 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 3 2024-08-18T13:39:31.575 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:31.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f4781cb5700 1 Processor -- start 2024-08-18T13:39:31.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f4781cb5700 1 -- start start 2024-08-18T13:39:31.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f4781cb5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c0673d0 0x7f477c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f4781cb5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f477c0c36a0 0x7f477c0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f4781cb5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c15a5a0 con 0x7f477c153600 2024-08-18T13:39:31.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f4781cb5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0c98e0 con 0x7f477c0c36a0 2024-08-18T13:39:31.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f4781cb5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0c94d0 con 0x7f477c0673d0 2024-08-18T13:39:31.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f47814b4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f477c0c36a0 0x7f477c0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f47814b4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f477c0c36a0 0x7f477c0c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53434/0 (socket says 172.21.15.118:53434) 2024-08-18T13:39:31.682 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f47814b4700 1 -- 172.21.15.118:0/604753400 learned_addr learned my addr 172.21.15.118:0/604753400 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:31.682 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.674+0000 7f477bfff700 1 --2- 172.21.15.118:0/604753400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c0673d0 0x7f477c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f47814b4700 1 -- 172.21.15.118:0/604753400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c0673d0 msgr2=0x7f477c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:31.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f47814b4700 1 --2- 172.21.15.118:0/604753400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c0673d0 0x7f477c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f47814b4700 1 -- 172.21.15.118:0/604753400 >> v1:172.21.15.178:6789/0 conn(0x7f477c153600 legacy=0x7f477c0c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f477b7fe700 1 -- 172.21.15.118:0/604753400 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441347637 0 0) 0x7f477c15a5a0 con 0x7f477c153600 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f47814b4700 1 -- 172.21.15.118:0/604753400 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0c4020 con 0x7f477c0c36a0 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f477bfff700 1 --2- 172.21.15.118:0/604753400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c0673d0 0x7f477c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f47814b4700 1 --2- 172.21.15.118:0/604753400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0c36a0 0x7f477c0c3ae0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4774009f20 tx=0x7f4774009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870ac3e18b5ce3b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f477b7fe700 1 -- 172.21.15.118:0/604753400 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400e070 con 0x7f477c0c36a0 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f477b7fe700 1 -- 172.21.15.118:0/604753400 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400ba60 con 0x7f477c0c36a0 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f477b7fe700 1 -- 172.21.15.118:0/604753400 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740092e0 con 0x7f477c0c36a0 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 -- 172.21.15.118:0/604753400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0c36a0 msgr2=0x7f477c0c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 --2- 172.21.15.118:0/604753400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0c36a0 0x7f477c0c3ae0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4774009f20 tx=0x7f4774009840 comp rx=0 tx=0).stop 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 -- 172.21.15.118:0/604753400 shutdown_connections 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 --2- 172.21.15.118:0/604753400 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0c36a0 0x7f477c0c3ae0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 --2- 172.21.15.118:0/604753400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c0673d0 0x7f477c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 -- 172.21.15.118:0/604753400 >> 172.21.15.118:0/604753400 conn(0x7f477c06f4d0 msgr2=0x7f477c0c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:31.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 -- 172.21.15.118:0/604753400 shutdown_connections 2024-08-18T13:39:31.685 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 -- 172.21.15.118:0/604753400 wait complete. 2024-08-18T13:39:31.685 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.678+0000 7f4781cb5700 1 Processor -- start 2024-08-18T13:39:31.685 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 -- start start 2024-08-18T13:39:31.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0673d0 0x7f477c16a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c36a0 0x7f477c14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c153600 0x7f477c158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1562a0 con 0x7f477c0673d0 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c14f9d0 con 0x7f477c0c36a0 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c16a380 con 0x7f477c153600 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0673d0 0x7f477c16a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0673d0 0x7f477c16a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53436/0 (socket says 172.21.15.118:53436) 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 -- 172.21.15.118:0/2982266036 learned_addr learned my addr 172.21.15.118:0/2982266036 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f47814b4700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c153600 0x7f477c158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f477bfff700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c36a0 0x7f477c14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 -- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c153600 msgr2=0x7f477c158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c153600 0x7f477c158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 -- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c36a0 msgr2=0x7f477c14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c36a0 0x7f477c14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 -- 172.21.15.118:0/2982266036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c009420 con 0x7f477c0673d0 2024-08-18T13:39:31.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4780cb3700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0673d0 0x7f477c16a940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f476400c530 tx=0x7f476400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:31.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4778ff9700 1 -- 172.21.15.118:0/2982266036 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011590 con 0x7f477c0673d0 2024-08-18T13:39:31.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4778ff9700 1 -- 172.21.15.118:0/2982266036 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764011710 con 0x7f477c0673d0 2024-08-18T13:39:31.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4774009ae0 con 0x7f477c0673d0 2024-08-18T13:39:31.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4778ff9700 1 -- 172.21.15.118:0/2982266036 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011ad0 con 0x7f477c0673d0 2024-08-18T13:39:31.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.682+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c16a0c0 con 0x7f477c0673d0 2024-08-18T13:39:31.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.686+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c0713a0 con 0x7f477c0673d0 2024-08-18T13:39:31.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.686+0000 7f4778ff9700 1 -- 172.21.15.118:0/2982266036 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f476401b3b0 con 0x7f477c0673d0 2024-08-18T13:39:31.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.686+0000 7f4778ff9700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f475c060890 0x7f475c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.686+0000 7f477bfff700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f475c060890 0x7f475c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.686+0000 7f4778ff9700 1 -- 172.21.15.118:0/2982266036 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f476407bc90 con 0x7f477c0673d0 2024-08-18T13:39:31.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.686+0000 7f477bfff700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f475c060890 0x7f475c062d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f477c0c1d60 tx=0x7f476c0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:31.696 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.690+0000 7f4778ff9700 1 -- 172.21.15.118:0/2982266036 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f476404c430 con 0x7f477c0673d0 2024-08-18T13:39:31.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.858+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f477c0c8750 con 0x7f477c0673d0 2024-08-18T13:39:31.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.858+0000 7f4778ff9700 1 -- 172.21.15.118:0/2982266036 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f4764017090 con 0x7f477c0673d0 2024-08-18T13:39:31.865 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:31.866 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":17,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:23.767627+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f475c060890 msgr2=0x7f475c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f475c060890 0x7f475c062d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f477c0c1d60 tx=0x7f476c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0673d0 msgr2=0x7f477c16a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0673d0 0x7f477c16a940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f476400c530 tx=0x7f476400c840 comp rx=0 tx=0).stop 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 shutdown_connections 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f475c060890 0x7f475c062d40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c153600 0x7f477c158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c36a0 0x7f477c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 --2- 172.21.15.118:0/2982266036 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f477c0673d0 0x7f477c16a940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 >> 172.21.15.118:0/2982266036 conn(0x7f477c06f4d0 msgr2=0x7f477c0c7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:31.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 shutdown_connections 2024-08-18T13:39:31.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.862+0000 7f4781cb5700 1 -- 172.21.15.118:0/2982266036 wait complete. 2024-08-18T13:39:31.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T13:39:21.246221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-18T13:39:31.880 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T13:39:31.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1650d700 1 Processor -- start 2024-08-18T13:39:31.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1650d700 1 -- start start 2024-08-18T13:39:31.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1650d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faa10069bf0 0x7faa100c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1650d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa100c88f0 0x7faa100673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:31.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1650d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa100c38b0 con 0x7faa1006db00 2024-08-18T13:39:31.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1650d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa10151a50 con 0x7faa10069bf0 2024-08-18T13:39:31.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1650d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1006b820 con 0x7faa100c88f0 2024-08-18T13:39:31.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa14d0a700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7faa1006db00 0x7faa1014b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37200/0 (socket says 172.21.15.118:37200) 2024-08-18T13:39:31.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa14d0a700 1 -- 172.21.15.118:0/1936381854 learned_addr learned my addr 172.21.15.118:0/1936381854 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1550b700 1 --2- 172.21.15.118:0/1936381854 >> v2:172.21.15.118:3300/0 conn(0x7faa10069bf0 0x7faa100c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa15d0c700 1 --2- 172.21.15.118:0/1936381854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa100c88f0 0x7faa100673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa07fff700 1 -- 172.21.15.118:0/1936381854 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143586249 0 0) 0x7faa100c38b0 con 0x7faa1006db00 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa07fff700 1 -- 172.21.15.118:0/1936381854 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9fc003570 con 0x7faa1006db00 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1550b700 1 -- 172.21.15.118:0/1936381854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa100c88f0 msgr2=0x7faa100673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1550b700 1 --2- 172.21.15.118:0/1936381854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa100c88f0 0x7faa100673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1550b700 1 -- 172.21.15.118:0/1936381854 >> v1:172.21.15.118:6789/0 conn(0x7faa1006db00 legacy=0x7faa1014b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.986+0000 7faa1550b700 1 -- 172.21.15.118:0/1936381854 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa10151790 con 0x7faa10069bf0 2024-08-18T13:39:31.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1550b700 1 --2- 172.21.15.118:0/1936381854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa10069bf0 0x7faa100c3d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00aa90 tx=0x7faa0c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3c7c0906f62de5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:31.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa07fff700 1 -- 172.21.15.118:0/1936381854 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00c970 con 0x7faa10069bf0 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa07fff700 1 -- 172.21.15.118:0/1936381854 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c00cad0 con 0x7faa10069bf0 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa07fff700 1 -- 172.21.15.118:0/1936381854 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00ce90 con 0x7faa10069bf0 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 -- 172.21.15.118:0/1936381854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa10069bf0 msgr2=0x7faa100c3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 --2- 172.21.15.118:0/1936381854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa10069bf0 0x7faa100c3d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00aa90 tx=0x7faa0c00a330 comp rx=0 tx=0).stop 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 -- 172.21.15.118:0/1936381854 shutdown_connections 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 --2- 172.21.15.118:0/1936381854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa100c88f0 0x7faa100673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 --2- 172.21.15.118:0/1936381854 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa10069bf0 0x7faa100c3d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 -- 172.21.15.118:0/1936381854 >> 172.21.15.118:0/1936381854 conn(0x7faa1006c750 msgr2=0x7faa100c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 -- 172.21.15.118:0/1936381854 shutdown_connections 2024-08-18T13:39:31.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 -- 172.21.15.118:0/1936381854 wait complete. 2024-08-18T13:39:31.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 Processor -- start 2024-08-18T13:39:31.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.990+0000 7faa1650d700 1 -- start start 2024-08-18T13:39:32.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa10069bf0 0x7faa10161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa1006db00 0x7faa100c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa100c88f0 0x7faa101465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa10161aa0 con 0x7faa100c88f0 2024-08-18T13:39:32.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100649d0 con 0x7faa1006db00 2024-08-18T13:39:32.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa10145880 con 0x7faa10069bf0 2024-08-18T13:39:32.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa100c88f0 0x7faa101465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa100c88f0 0x7faa101465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53442/0 (socket says 172.21.15.118:53442) 2024-08-18T13:39:32.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 -- 172.21.15.118:0/800210920 learned_addr learned my addr 172.21.15.118:0/800210920 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1550b700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa10069bf0 0x7faa10161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa14d0a700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa1006db00 0x7faa100c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 -- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa10069bf0 msgr2=0x7faa10161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa10069bf0 0x7faa10161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 -- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa1006db00 msgr2=0x7faa100c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa1006db00 0x7faa100c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 -- 172.21.15.118:0/800210920 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c00a5d0 con 0x7faa100c88f0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa15d0c700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa100c88f0 0x7faa101465b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa08010fb0 tx=0x7faa0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa057fa700 1 -- 172.21.15.118:0/800210920 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800ee70 con 0x7faa100c88f0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa057fa700 1 -- 172.21.15.118:0/800210920 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa08012430 con 0x7faa100c88f0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa10146120 con 0x7faa100c88f0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa057fa700 1 -- 172.21.15.118:0/800210920 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa080127f0 con 0x7faa100c88f0 2024-08-18T13:39:32.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.994+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa100c0d00 con 0x7faa100c88f0 2024-08-18T13:39:32.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.998+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa10069180 con 0x7faa100c88f0 2024-08-18T13:39:32.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.998+0000 7faa057fa700 1 -- 172.21.15.118:0/800210920 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7faa08012ba0 con 0x7faa100c88f0 2024-08-18T13:39:32.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.998+0000 7faa057fa700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa9f4060860 0x7fa9f4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.998+0000 7faa057fa700 1 -- 172.21.15.118:0/800210920 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7faa08016070 con 0x7faa100c88f0 2024-08-18T13:39:32.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.998+0000 7faa1550b700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa9f4060860 0x7fa9f4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:31.998+0000 7faa1550b700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa9f4060860 0x7fa9f4062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00a5a0 tx=0x7faa0c01a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.006+0000 7faa057fa700 1 -- 172.21.15.118:0/800210920 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faa10069180 con 0x7faa100c88f0 2024-08-18T13:39:32.172 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.166+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa10069180 con 0x7faa100c88f0 2024-08-18T13:39:32.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.166+0000 7faa057fa700 1 -- 172.21.15.118:0/800210920 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa10069180 con 0x7faa100c88f0 2024-08-18T13:39:32.173 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa9f4060860 msgr2=0x7fa9f4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa9f4060860 0x7fa9f4062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00a5a0 tx=0x7faa0c01a000 comp rx=0 tx=0).stop 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa100c88f0 msgr2=0x7faa101465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa100c88f0 0x7faa101465b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa08010fb0 tx=0x7faa0800b040 comp rx=0 tx=0).stop 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 shutdown_connections 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa9f4060860 0x7fa9f4062d10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa100c88f0 0x7faa101465b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa1006db00 0x7faa100c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 --2- 172.21.15.118:0/800210920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa10069bf0 0x7faa10161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 >> 172.21.15.118:0/800210920 conn(0x7faa1006c750 msgr2=0x7faa1014f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:32.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 shutdown_connections 2024-08-18T13:39:32.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.170+0000 7faa1650d700 1 -- 172.21.15.118:0/800210920 wait complete. 2024-08-18T13:39:32.187 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-18T13:39:32.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.282+0000 7fbc9f59e700 1 Processor -- start 2024-08-18T13:39:32.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- start start 2024-08-18T13:39:32.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca00c8200 0x7fbca00673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbca006f2f0 0x7fbca00c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca006f770 con 0x7fbca0153690 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca015a640 con 0x7fbca006f2f0 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca006ee60 con 0x7fbca00c8200 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9e59c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbca0153690 0x7fbca00c22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37210/0 (socket says 172.21.15.118:37210) 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9e59c700 1 -- 172.21.15.118:0/668203223 learned_addr learned my addr 172.21.15.118:0/668203223 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9dd9b700 1 --2- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca00c8200 0x7fbca00673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9ed9d700 1 --2- 172.21.15.118:0/668203223 >> v2:172.21.15.178:3300/0 conn(0x7fbca006f2f0 0x7fbca00c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9d59a700 1 -- 172.21.15.118:0/668203223 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511939721 0 0) 0x7fbc98001490 con 0x7fbca0153690 2024-08-18T13:39:32.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9d59a700 1 -- 172.21.15.118:0/668203223 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc8c003570 con 0x7fbca0153690 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9d59a700 1 -- 172.21.15.118:0/668203223 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757634045 0 0) 0x7fbc8c003570 con 0x7fbca0153690 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9ed9d700 1 -- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca00c8200 msgr2=0x7fbca00673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9ed9d700 1 --2- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca00c8200 0x7fbca00673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9ed9d700 1 -- 172.21.15.118:0/668203223 >> v1:172.21.15.118:6789/0 conn(0x7fbca0153690 legacy=0x7fbca00c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9ed9d700 1 -- 172.21.15.118:0/668203223 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0153d10 con 0x7fbca006f2f0 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9ed9d700 1 --2- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca006f2f0 0x7fbca00c29b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098e0 tx=0x7fbc900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468ddcbcfcf7d49a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9d59a700 1 -- 172.21.15.118:0/668203223 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98001980 con 0x7fbca006f2f0 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9d59a700 1 -- 172.21.15.118:0/668203223 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc90011420 con 0x7fbca006f2f0 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9d59a700 1 -- 172.21.15.118:0/668203223 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900117a0 con 0x7fbca006f2f0 2024-08-18T13:39:32.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca006f2f0 msgr2=0x7fbca00c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca006f2f0 0x7fbca00c29b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098e0 tx=0x7fbc900091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- 172.21.15.118:0/668203223 shutdown_connections 2024-08-18T13:39:32.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca006f2f0 0x7fbca00c29b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/668203223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca00c8200 0x7fbca00673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- 172.21.15.118:0/668203223 >> 172.21.15.118:0/668203223 conn(0x7fbca006c020 msgr2=0x7fbca00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:32.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- 172.21.15.118:0/668203223 shutdown_connections 2024-08-18T13:39:32.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.286+0000 7fbc9f59e700 1 -- 172.21.15.118:0/668203223 wait complete. 2024-08-18T13:39:32.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 Processor -- start 2024-08-18T13:39:32.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 -- start start 2024-08-18T13:39:32.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca006f2f0 0x7fbca016a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca00c8200 0x7fbca01699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbca0153690 0x7fbca0158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0169520 con 0x7fbca0153690 2024-08-18T13:39:32.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0168f60 con 0x7fbca00c8200 2024-08-18T13:39:32.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca016d310 con 0x7fbca006f2f0 2024-08-18T13:39:32.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbca0153690 0x7fbca0158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbca0153690 0x7fbca0158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53452/0 (socket says 172.21.15.118:53452) 2024-08-18T13:39:32.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 -- 172.21.15.118:0/2304341674 learned_addr learned my addr 172.21.15.118:0/2304341674 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:32.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9dd9b700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca00c8200 0x7fbca01699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9e59c700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca006f2f0 0x7fbca016a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 -- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca006f2f0 msgr2=0x7fbca016a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca006f2f0 0x7fbca016a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 -- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca00c8200 msgr2=0x7fbca01699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca00c8200 0x7fbca01699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 -- 172.21.15.118:0/2304341674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90009420 con 0x7fbca0153690 2024-08-18T13:39:32.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc9ed9d700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbca0153690 0x7fbca0158440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbca005eb30 tx=0x7fbc90016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc967fc700 1 -- 172.21.15.118:0/2304341674 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9001cc40 con 0x7fbca0153690 2024-08-18T13:39:32.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.290+0000 7fbc967fc700 1 -- 172.21.15.118:0/2304341674 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9001cda0 con 0x7fbca0153690 2024-08-18T13:39:32.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca0154000 con 0x7fbca0153690 2024-08-18T13:39:32.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc967fc700 1 -- 172.21.15.118:0/2304341674 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9001a6a0 con 0x7fbca0153690 2024-08-18T13:39:32.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca01691b0 con 0x7fbca0153690 2024-08-18T13:39:32.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca01696d0 con 0x7fbca0153690 2024-08-18T13:39:32.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc967fc700 1 -- 172.21.15.118:0/2304341674 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbc9001aa10 con 0x7fbca0153690 2024-08-18T13:39:32.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc967fc700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbc80060890 0x7fbc80062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc967fc700 1 -- 172.21.15.118:0/2304341674 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbc9007df90 con 0x7fbca0153690 2024-08-18T13:39:32.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc9e59c700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbc80060890 0x7fbc80062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.294+0000 7fbc9e59c700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbc80060890 0x7fbc80062d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc98001450 tx=0x7fbc98007f30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.302+0000 7fbc967fc700 1 -- 172.21.15.118:0/2304341674 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbc900ae480 con 0x7fbca0153690 2024-08-18T13:39:32.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.470+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbca00713a0 con 0x7fbca0153690 2024-08-18T13:39:32.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.470+0000 7fbc967fc700 1 -- 172.21.15.118:0/2304341674 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbc90029060 con 0x7fbca0153690 2024-08-18T13:39:32.476 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:32.476 INFO:teuthology.orchestra.run.smithi118.stdout:[ 2024-08-18T13:39:32.477 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-08-18T13:39:32.477 INFO:teuthology.orchestra.run.smithi118.stdout: "poolnum": 1, 2024-08-18T13:39:32.477 INFO:teuthology.orchestra.run.smithi118.stdout: "poolname": ".mgr" 2024-08-18T13:39:32.477 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-08-18T13:39:32.477 INFO:teuthology.orchestra.run.smithi118.stdout:] 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbc80060890 msgr2=0x7fbc80062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbc80060890 0x7fbc80062d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc98001450 tx=0x7fbc98007f30 comp rx=0 tx=0).stop 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbca0153690 msgr2=0x7fbca0158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbca0153690 0x7fbca0158440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbca005eb30 tx=0x7fbc90016000 comp rx=0 tx=0).stop 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 shutdown_connections 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbc80060890 0x7fbc80062d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbca0153690 0x7fbca0158440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbca00c8200 0x7fbca01699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 --2- 172.21.15.118:0/2304341674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbca006f2f0 0x7fbca016a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 >> 172.21.15.118:0/2304341674 conn(0x7fbca006c020 msgr2=0x7fbca006dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:32.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 shutdown_connections 2024-08-18T13:39:32.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.474+0000 7fbc9f59e700 1 -- 172.21.15.118:0/2304341674 wait complete. 2024-08-18T13:39:32.494 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-18T13:39:32.494 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-18T13:39:32.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.602+0000 7f6aa8d98700 1 Processor -- start 2024-08-18T13:39:32.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.602+0000 7f6aa8d98700 1 -- start start 2024-08-18T13:39:32.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa40c87c0 0x7f6aa4068e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6aa40c8b90 0x7f6aa40693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa40689e0 con 0x7f6aa41551a0 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa4156190 con 0x7f6aa40c8b90 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4155630 con 0x7f6aa40c87c0 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6aa40c8b90 0x7f6aa40693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6aa40c8b90 0x7f6aa40693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53456/0 (socket says 172.21.15.118:53456) 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 -- 172.21.15.118:0/1249953069 learned_addr learned my addr 172.21.15.118:0/1249953069 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa37fe700 1 --2- 172.21.15.118:0/1249953069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa40c87c0 0x7f6aa4068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 -- 172.21.15.118:0/1249953069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa40c87c0 msgr2=0x7f6aa4068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 --2- 172.21.15.118:0/1249953069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa40c87c0 0x7f6aa4068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 -- 172.21.15.118:0/1249953069 >> v1:172.21.15.178:6789/0 conn(0x7f6aa41551a0 legacy=0x7f6aa40c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 -- 172.21.15.118:0/1249953069 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa41558f0 con 0x7f6aa40c8b90 2024-08-18T13:39:32.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa37fe700 1 --2- 172.21.15.118:0/1249953069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa40c87c0 0x7f6aa4068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:32.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa3fff700 1 --2- 172.21.15.118:0/1249953069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c8b90 0x7f6aa40693b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009f20 tx=0x7f6a9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48b8b51b8a45427 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa27fc700 1 -- 172.21.15.118:0/1249953069 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00e070 con 0x7f6aa40c8b90 2024-08-18T13:39:32.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa27fc700 1 -- 172.21.15.118:0/1249953069 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c00bb10 con 0x7f6aa40c8b90 2024-08-18T13:39:32.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa27fc700 1 -- 172.21.15.118:0/1249953069 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0092e0 con 0x7f6aa40c8b90 2024-08-18T13:39:32.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- 172.21.15.118:0/1249953069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c8b90 msgr2=0x7f6aa40693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/1249953069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c8b90 0x7f6aa40693b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009f20 tx=0x7f6a9c009840 comp rx=0 tx=0).stop 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- 172.21.15.118:0/1249953069 shutdown_connections 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/1249953069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c8b90 0x7f6aa40693b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/1249953069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa40c87c0 0x7f6aa4068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- 172.21.15.118:0/1249953069 >> 172.21.15.118:0/1249953069 conn(0x7f6aa40673d0 msgr2=0x7f6aa40677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- 172.21.15.118:0/1249953069 shutdown_connections 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.606+0000 7f6aa8d98700 1 -- 172.21.15.118:0/1249953069 wait complete. 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 Processor -- start 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 -- start start 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c87c0 0x7f6aa40c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa40c8b90 0x7f6aa4165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa41551a0 0x7f6aa41410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40cacf0 con 0x7f6aa40c87c0 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4157ae0 con 0x7f6aa40c8b90 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4157dc0 con 0x7f6aa41551a0 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c87c0 0x7f6aa40c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c87c0 0x7f6aa40c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53470/0 (socket says 172.21.15.118:53470) 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 -- 172.21.15.118:0/255028848 learned_addr learned my addr 172.21.15.118:0/255028848 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa3fff700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa41551a0 0x7f6aa41410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa2ffd700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa40c8b90 0x7f6aa4165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 -- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa41551a0 msgr2=0x7f6aa41410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa41551a0 0x7f6aa41410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 -- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa40c8b90 msgr2=0x7f6aa4165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa40c8b90 0x7f6aa4165160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 -- 172.21.15.118:0/255028848 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c009ae0 con 0x7f6aa40c87c0 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa37fe700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c87c0 0x7f6aa40c73b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800e8d0 tx=0x7f6a98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6a8b7fe700 1 -- 172.21.15.118:0/255028848 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98013410 con 0x7f6aa40c87c0 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6a8b7fe700 1 -- 172.21.15.118:0/255028848 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98013570 con 0x7f6aa40c87c0 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa41580a0 con 0x7f6aa40c87c0 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6a8b7fe700 1 -- 172.21.15.118:0/255028848 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98013910 con 0x7f6aa40c87c0 2024-08-18T13:39:32.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.610+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa40c9f30 con 0x7f6aa40c87c0 2024-08-18T13:39:32.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.614+0000 7f6a8b7fe700 1 -- 172.21.15.118:0/255028848 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6a98013cc0 con 0x7f6aa40c87c0 2024-08-18T13:39:32.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.614+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa4071410 con 0x7f6aa40c87c0 2024-08-18T13:39:32.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.614+0000 7f6a8b7fe700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6a90060860 0x7f6a90062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:32.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.614+0000 7f6a8b7fe700 1 -- 172.21.15.118:0/255028848 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6a9807c5e0 con 0x7f6aa40c87c0 2024-08-18T13:39:32.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.614+0000 7f6aa2ffd700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6a90060860 0x7f6a90062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:32.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.614+0000 7f6aa2ffd700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6a90060860 0x7f6a90062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4149760 tx=0x7f6a94007ea0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:32.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.622+0000 7f6a8b7fe700 1 -- 172.21.15.118:0/255028848 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6a9804cd80 con 0x7f6aa40c87c0 2024-08-18T13:39:32.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:32.782+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6aa40694d0 con 0x7f6aa40c87c0 2024-08-18T13:39:33.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.358+0000 7f6a8b7fe700 1 -- 172.21.15.118:0/255028848 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6a980503a0 con 0x7f6aa40c87c0 2024-08-18T13:39:33.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6a90060860 msgr2=0x7f6a90062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:33.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6a90060860 0x7f6a90062d10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4149760 tx=0x7f6a94007ea0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c87c0 msgr2=0x7f6aa40c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:33.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c87c0 0x7f6aa40c73b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800e8d0 tx=0x7f6a98008040 comp rx=0 tx=0).stop 2024-08-18T13:39:33.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 shutdown_connections 2024-08-18T13:39:33.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6a90060860 0x7f6a90062d10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6aa41551a0 0x7f6aa41410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6aa40c8b90 0x7f6aa4165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 --2- 172.21.15.118:0/255028848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6aa40c87c0 0x7f6aa40c73b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 >> 172.21.15.118:0/255028848 conn(0x7f6aa40673d0 msgr2=0x7f6aa4153b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:33.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 shutdown_connections 2024-08-18T13:39:33.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.370+0000 7f6aa8d98700 1 -- 172.21.15.118:0/255028848 wait complete. 2024-08-18T13:39:33.376 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_metadata' created 2024-08-18T13:39:33.387 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-08-18T13:39:33.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.498+0000 7f210cd5a700 1 Processor -- start 2024-08-18T13:39:33.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.498+0000 7f210cd5a700 1 -- start start 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f210806db00 0x7f2108158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f21080c3590 0x7f2108159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2108068c50 con 0x7f21080bec20 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210805eb10 con 0x7f210806db00 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210806ab90 con 0x7f21080c3590 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f210806db00 0x7f2108158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f210806db00 0x7f2108158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53484/0 (socket says 172.21.15.118:53484) 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 -- 172.21.15.118:0/3079948980 learned_addr learned my addr 172.21.15.118:0/3079948980 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2107fff700 1 --2- 172.21.15.118:0/3079948980 >> v2:172.21.15.178:3300/0 conn(0x7f21080c3590 0x7f2108159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 -- 172.21.15.118:0/3079948980 >> v2:172.21.15.178:3300/0 conn(0x7f21080c3590 msgr2=0x7f2108159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 --2- 172.21.15.118:0/3079948980 >> v2:172.21.15.178:3300/0 conn(0x7f21080c3590 0x7f2108159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 -- 172.21.15.118:0/3079948980 >> v1:172.21.15.178:6789/0 conn(0x7f21080bec20 legacy=0x7f2108069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:33.509 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 -- 172.21.15.118:0/3079948980 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210806a8b0 con 0x7f210806db00 2024-08-18T13:39:33.509 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f2106ffd700 1 --2- 172.21.15.118:0/3079948980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f210806db00 0x7f2108158bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f210806e090 tx=0x7f20f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ea16cf5f5a94dd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:33.509 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f21067fc700 1 -- 172.21.15.118:0/3079948980 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000b860 con 0x7f210806db00 2024-08-18T13:39:33.509 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f21067fc700 1 -- 172.21.15.118:0/3079948980 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f000b9c0 con 0x7f210806db00 2024-08-18T13:39:33.509 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f21067fc700 1 -- 172.21.15.118:0/3079948980 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000bd60 con 0x7f210806db00 2024-08-18T13:39:33.509 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- 172.21.15.118:0/3079948980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f210806db00 msgr2=0x7f2108158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:33.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3079948980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f210806db00 0x7f2108158bd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f210806e090 tx=0x7f20f0009250 comp rx=0 tx=0).stop 2024-08-18T13:39:33.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- 172.21.15.118:0/3079948980 shutdown_connections 2024-08-18T13:39:33.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3079948980 >> v2:172.21.15.178:3300/0 conn(0x7f21080c3590 0x7f2108159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3079948980 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f210806db00 0x7f2108158bd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- 172.21.15.118:0/3079948980 >> 172.21.15.118:0/3079948980 conn(0x7f210806c750 msgr2=0x7f21080c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:33.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- 172.21.15.118:0/3079948980 shutdown_connections 2024-08-18T13:39:33.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.502+0000 7f210cd5a700 1 -- 172.21.15.118:0/3079948980 wait complete. 2024-08-18T13:39:33.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 Processor -- start 2024-08-18T13:39:33.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 -- start start 2024-08-18T13:39:33.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210806db00 0x7f210814adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:33.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21080bec20 0x7f210814ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:33.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21080c3590 0x7f210814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2108153d40 con 0x7f21080c3590 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2108157de0 con 0x7f210806db00 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f210cd5a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2108156170 con 0x7f21080bec20 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21080c3590 0x7f210814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21080c3590 0x7f210814d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53500/0 (socket says 172.21.15.118:53500) 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 -- 172.21.15.118:0/3121410157 learned_addr learned my addr 172.21.15.118:0/3121410157 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f21077fe700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210806db00 0x7f210814adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2106ffd700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21080bec20 0x7f210814ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 -- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21080bec20 msgr2=0x7f210814ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21080bec20 0x7f210814ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 -- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210806db00 msgr2=0x7f210814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210806db00 0x7f210814adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:33.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.506+0000 7f2107fff700 1 -- 172.21.15.118:0/3121410157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00094f0 con 0x7f21080c3590 2024-08-18T13:39:33.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f2107fff700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21080c3590 0x7f210814d3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2100007f50 tx=0x7f2100009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:33.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f20ff7fe700 1 -- 172.21.15.118:0/3121410157 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210000eba0 con 0x7f21080c3590 2024-08-18T13:39:33.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f20ff7fe700 1 -- 172.21.15.118:0/3121410157 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210000ed20 con 0x7f21080c3590 2024-08-18T13:39:33.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2108154790 con 0x7f21080c3590 2024-08-18T13:39:33.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f20ff7fe700 1 -- 172.21.15.118:0/3121410157 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100014670 con 0x7f21080c3590 2024-08-18T13:39:33.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2108153380 con 0x7f21080c3590 2024-08-18T13:39:33.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2108071410 con 0x7f21080c3590 2024-08-18T13:39:33.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f20ff7fe700 1 -- 172.21.15.118:0/3121410157 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f21000149e0 con 0x7f21080c3590 2024-08-18T13:39:33.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f20ff7fe700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f20e8060890 0x7f20e8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:33.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.510+0000 7f20ff7fe700 1 -- 172.21.15.118:0/3121410157 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f210007cf60 con 0x7f21080c3590 2024-08-18T13:39:33.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.514+0000 7f21077fe700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f20e8060890 0x7f20e8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:33.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.514+0000 7f21077fe700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f20e8060890 0x7f20e8062d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20f8003a80 tx=0x7f20f8007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:33.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.518+0000 7f20ff7fe700 1 -- 172.21.15.118:0/3121410157 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f210004d590 con 0x7f21080c3590 2024-08-18T13:39:33.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:33.690+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f21081562b0 con 0x7f21080c3590 2024-08-18T13:39:34.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.366+0000 7f20ff7fe700 1 -- 172.21.15.118:0/3121410157 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f2100050bb0 con 0x7f21080c3590 2024-08-18T13:39:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.374+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f20e8060890 msgr2=0x7f20e8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.374+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f20e8060890 0x7f20e8062d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20f8003a80 tx=0x7f20f8007ec0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.374+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21080c3590 msgr2=0x7f210814d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.374+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21080c3590 0x7f210814d3a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2100007f50 tx=0x7f2100009860 comp rx=0 tx=0).stop 2024-08-18T13:39:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.378+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 shutdown_connections 2024-08-18T13:39:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.378+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f20e8060890 0x7f20e8062d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.378+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f21080c3590 0x7f210814d3a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.378+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21080bec20 0x7f210814ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.378+0000 7f210cd5a700 1 --2- 172.21.15.118:0/3121410157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f210806db00 0x7f210814adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.378+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 >> 172.21.15.118:0/3121410157 conn(0x7f210806c750 msgr2=0x7f210805dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:34.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.382+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 shutdown_connections 2024-08-18T13:39:34.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.382+0000 7f210cd5a700 1 -- 172.21.15.118:0/3121410157 wait complete. 2024-08-18T13:39:34.397 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_data' created 2024-08-18T13:39:34.405 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-18T13:39:34.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.506+0000 7f3eb2059700 1 Processor -- start 2024-08-18T13:39:34.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.506+0000 7f3eb2059700 1 -- start start 2024-08-18T13:39:34.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.506+0000 7f3eb2059700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3eac06c3d0 0x7f3eac14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:34.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.506+0000 7f3eb2059700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac069f90 0x7f3eac14c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:34.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.506+0000 7f3eb2059700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac068dd0 con 0x7f3eac0c8f80 2024-08-18T13:39:34.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.506+0000 7f3eb2059700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac152ab0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.506+0000 7f3eb2059700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac152d50 con 0x7f3eac069f90 2024-08-18T13:39:34.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac069f90 0x7f3eac14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:34.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac069f90 0x7f3eac14c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:38156/0 (socket says 172.21.15.118:38156) 2024-08-18T13:39:34.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 -- 172.21.15.118:0/417756235 learned_addr learned my addr 172.21.15.118:0/417756235 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:34.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb1057700 1 --2- 172.21.15.118:0/417756235 >> v2:172.21.15.178:3300/0 conn(0x7f3eac06c3d0 0x7f3eac14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:34.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 -- 172.21.15.118:0/417756235 >> v2:172.21.15.178:3300/0 conn(0x7f3eac06c3d0 msgr2=0x7f3eac14bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:34.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 --2- 172.21.15.118:0/417756235 >> v2:172.21.15.178:3300/0 conn(0x7f3eac06c3d0 0x7f3eac14bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 -- 172.21.15.118:0/417756235 >> v1:172.21.15.178:6789/0 conn(0x7f3eac0c8f80 legacy=0x7f3eac069260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:34.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 -- 172.21.15.118:0/417756235 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac068780 con 0x7f3eac069f90 2024-08-18T13:39:34.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb0856700 1 --2- 172.21.15.118:0/417756235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac069f90 0x7f3eac14c280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0099b0 tx=0x7f3e9c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724af527e520130b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3ea3fff700 1 -- 172.21.15.118:0/417756235 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c009ea0 con 0x7f3eac069f90 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3ea3fff700 1 -- 172.21.15.118:0/417756235 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00b860 con 0x7f3eac069f90 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3ea3fff700 1 -- 172.21.15.118:0/417756235 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00bc00 con 0x7f3eac069f90 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 -- 172.21.15.118:0/417756235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac069f90 msgr2=0x7f3eac14c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 --2- 172.21.15.118:0/417756235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac069f90 0x7f3eac14c280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0099b0 tx=0x7f3e9c009250 comp rx=0 tx=0).stop 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 -- 172.21.15.118:0/417756235 shutdown_connections 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 --2- 172.21.15.118:0/417756235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac069f90 0x7f3eac14c280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 --2- 172.21.15.118:0/417756235 >> v2:172.21.15.178:3300/0 conn(0x7f3eac06c3d0 0x7f3eac14bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 -- 172.21.15.118:0/417756235 >> 172.21.15.118:0/417756235 conn(0x7f3eac0c5e00 msgr2=0x7f3eac0c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 -- 172.21.15.118:0/417756235 shutdown_connections 2024-08-18T13:39:34.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.510+0000 7f3eb2059700 1 -- 172.21.15.118:0/417756235 wait complete. 2024-08-18T13:39:34.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 Processor -- start 2024-08-18T13:39:34.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 -- start start 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eac069f90 0x7f3eac15fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eac06c3d0 0x7f3eac0c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac0c8f80 0x7f3eac0c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac14e430 con 0x7f3eac06c3d0 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac14f610 con 0x7f3eac069f90 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1637a0 con 0x7f3eac0c8f80 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eac06c3d0 0x7f3eac0c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eac06c3d0 0x7f3eac0c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54050/0 (socket says 172.21.15.118:54050) 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 -- 172.21.15.118:0/1601402431 learned_addr learned my addr 172.21.15.118:0/1601402431 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb1057700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eac069f90 0x7f3eac15fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:34.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb1858700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac0c8f80 0x7f3eac0c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:34.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 -- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac0c8f80 msgr2=0x7f3eac0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:34.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac0c8f80 0x7f3eac0c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 -- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eac069f90 msgr2=0x7f3eac15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:34.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eac069f90 0x7f3eac15fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:34.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 -- 172.21.15.118:0/1601402431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0094f0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb1858700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac0c8f80 0x7f3eac0c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:34.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb0856700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eac06c3d0 0x7f3eac0c43a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3eac152e90 tx=0x7f3e9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:34.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3ea17fa700 1 -- 172.21.15.118:0/1601402431 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00bbe0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3ea17fa700 1 -- 172.21.15.118:0/1601402431 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c011420 con 0x7f3eac06c3d0 2024-08-18T13:39:34.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac163a80 con 0x7f3eac06c3d0 2024-08-18T13:39:34.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3ea17fa700 1 -- 172.21.15.118:0/1601402431 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c0117c0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.514+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac14f2a0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.518+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac14f7a0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.518+0000 7f3ea17fa700 1 -- 172.21.15.118:0/1601402431 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3e9c011bb0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.518+0000 7f3ea17fa700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e90060860 0x7f3e90062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:34.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.518+0000 7f3eb1057700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e90060860 0x7f3e90062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:34.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.518+0000 7f3ea17fa700 1 -- 172.21.15.118:0/1601402431 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f3e9c07ea20 con 0x7f3eac06c3d0 2024-08-18T13:39:34.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.522+0000 7f3eb1057700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e90060860 0x7f3e90062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400f570 tx=0x7f3ea400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:34.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.526+0000 7f3ea17fa700 1 -- 172.21.15.118:0/1601402431 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3e9c04eeb0 con 0x7f3eac06c3d0 2024-08-18T13:39:34.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:34.730+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3eac071410 con 0x7f3eac06c3d0 2024-08-18T13:39:35.415 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.410+0000 7f3ea17fa700 1 -- 172.21.15.118:0/1601402431 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3e9c011e60 con 0x7f3eac06c3d0 2024-08-18T13:39:35.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e90060860 msgr2=0x7f3e90062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e90060860 0x7f3e90062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400f570 tx=0x7f3ea400b040 comp rx=0 tx=0).stop 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eac06c3d0 msgr2=0x7f3eac0c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eac06c3d0 0x7f3eac0c43a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3eac152e90 tx=0x7f3e9c017000 comp rx=0 tx=0).stop 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 shutdown_connections 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e90060860 0x7f3e90062d10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eac0c8f80 0x7f3eac0c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eac06c3d0 0x7f3eac0c43a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 --2- 172.21.15.118:0/1601402431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eac069f90 0x7f3eac15fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 >> 172.21.15.118:0/1601402431 conn(0x7f3eac0c5e00 msgr2=0x7f3eac14bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 shutdown_connections 2024-08-18T13:39:35.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.414+0000 7f3eb2059700 1 -- 172.21.15.118:0/1601402431 wait complete. 2024-08-18T13:39:35.420 INFO:teuthology.orchestra.run.smithi118.stderr:new fs with metadata pool 2 and data pool 3 2024-08-18T13:39:35.431 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-08-18T13:39:35.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-08-18T13:39:35.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.538+0000 7fba0d815700 1 Processor -- start 2024-08-18T13:39:35.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.538+0000 7fba0d815700 1 -- start start 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08156d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba080ac1b0 con 0x7fba080c78b0 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba08158e30 con 0x7fba081572b0 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba08157730 con 0x7fba08158a40 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba06ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fba080c78b0 0x7fba080c7c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:49220/0 (socket says 172.21.15.118:49220) 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba06ffd700 1 -- 172.21.15.118:0/3149781133 learned_addr learned my addr 172.21.15.118:0/3149781133 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba067fc700 1 --2- 172.21.15.118:0/3149781133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba077fe700 1 --2- 172.21.15.118:0/3149781133 >> v2:172.21.15.178:3300/0 conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba05ffb700 1 -- 172.21.15.118:0/3149781133 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567895028 0 0) 0x7fba080ac1b0 con 0x7fba080c78b0 2024-08-18T13:39:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba05ffb700 1 -- 172.21.15.118:0/3149781133 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f0003570 con 0x7fba080c78b0 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba05ffb700 1 -- 172.21.15.118:0/3149781133 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 562511454 0 0) 0x7fb9f0003570 con 0x7fba080c78b0 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba067fc700 1 -- 172.21.15.118:0/3149781133 >> v2:172.21.15.178:3300/0 conn(0x7fba081572b0 msgr2=0x7fba0814f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba067fc700 1 --2- 172.21.15.118:0/3149781133 >> v2:172.21.15.178:3300/0 conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba067fc700 1 -- 172.21.15.118:0/3149781133 >> v1:172.21.15.118:6789/0 conn(0x7fba080c78b0 legacy=0x7fba080c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba067fc700 1 -- 172.21.15.118:0/3149781133 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba080ac3e0 con 0x7fba08158a40 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba067fc700 1 --2- 172.21.15.118:0/3149781133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08156d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80098e0 tx=0x7fb9f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8437ae86b81b7f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba05ffb700 1 -- 172.21.15.118:0/3149781133 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc001470 con 0x7fba08158a40 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba05ffb700 1 -- 172.21.15.118:0/3149781133 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8011420 con 0x7fba08158a40 2024-08-18T13:39:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba05ffb700 1 -- 172.21.15.118:0/3149781133 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80117a0 con 0x7fba08158a40 2024-08-18T13:39:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- 172.21.15.118:0/3149781133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 msgr2=0x7fba08156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 --2- 172.21.15.118:0/3149781133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08156d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80098e0 tx=0x7fb9f80091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- 172.21.15.118:0/3149781133 shutdown_connections 2024-08-18T13:39:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 --2- 172.21.15.118:0/3149781133 >> v2:172.21.15.178:3300/0 conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 --2- 172.21.15.118:0/3149781133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08156d70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- 172.21.15.118:0/3149781133 >> 172.21.15.118:0/3149781133 conn(0x7fba080cc810 msgr2=0x7fba080c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:35.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- 172.21.15.118:0/3149781133 shutdown_connections 2024-08-18T13:39:35.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.542+0000 7fba0d815700 1 -- 172.21.15.118:0/3149781133 wait complete. 2024-08-18T13:39:35.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 Processor -- start 2024-08-18T13:39:35.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 -- start start 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba080c78b0 0x7fba081660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08068910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08166630 con 0x7fba081572b0 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0814d730 con 0x7fba080c78b0 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba0d815700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0814ea90 con 0x7fba08158a40 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54062/0 (socket says 172.21.15.118:54062) 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 -- 172.21.15.118:0/1095693356 learned_addr learned my addr 172.21.15.118:0/1095693356 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba077fe700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08068910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba06ffd700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba080c78b0 0x7fba081660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 -- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 msgr2=0x7fba08068910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08068910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 -- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba080c78b0 msgr2=0x7fba081660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba080c78b0 0x7fba081660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.546+0000 7fba067fc700 1 -- 172.21.15.118:0/1095693356 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f8009420 con 0x7fba081572b0 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fba067fc700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba081572b0 0x7fba0814f230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba08158f70 tx=0x7fb9f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fb9eeffd700 1 -- 172.21.15.118:0/1095693356 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801cba0 con 0x7fba081572b0 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fb9eeffd700 1 -- 172.21.15.118:0/1095693356 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f801cd00 con 0x7fba081572b0 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba081538c0 con 0x7fba081572b0 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fb9eeffd700 1 -- 172.21.15.118:0/1095693356 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801a6a0 con 0x7fba081572b0 2024-08-18T13:39:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0814ce00 con 0x7fba081572b0 2024-08-18T13:39:35.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba080ae340 con 0x7fba081572b0 2024-08-18T13:39:35.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.550+0000 7fb9eeffd700 1 -- 172.21.15.118:0/1095693356 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb9f801aa10 con 0x7fba081572b0 2024-08-18T13:39:35.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.554+0000 7fb9eeffd700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb9f4060860 0x7fb9f4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:35.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.554+0000 7fba06ffd700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb9f4060860 0x7fb9f4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:35.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.554+0000 7fb9eeffd700 1 -- 172.21.15.118:0/1095693356 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb9f807e5b0 con 0x7fba081572b0 2024-08-18T13:39:35.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.554+0000 7fba06ffd700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb9f4060860 0x7fb9f4062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001620 tx=0x7fb9fc0014c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:35.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.558+0000 7fb9eeffd700 1 -- 172.21.15.118:0/1095693356 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fba080ae340 con 0x7fba081572b0 2024-08-18T13:39:35.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:35.734+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fba081453b0 con 0x7fba081572b0 2024-08-18T13:39:36.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.386+0000 7fb9eeffd700 1 -- 172.21.15.118:0/1095693356 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fba081453b0 con 0x7fba081572b0 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb9f4060860 msgr2=0x7fb9f4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb9f4060860 0x7fb9f4062d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001620 tx=0x7fb9fc0014c0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba081572b0 msgr2=0x7fba0814f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba081572b0 0x7fba0814f230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba08158f70 tx=0x7fb9f8016000 comp rx=0 tx=0).stop 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 shutdown_connections 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb9f4060860 0x7fb9f4062d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba08158a40 0x7fba08068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba081572b0 0x7fba0814f230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 --2- 172.21.15.118:0/1095693356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba080c78b0 0x7fba081660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.390+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 >> 172.21.15.118:0/1095693356 conn(0x7fba080cc810 msgr2=0x7fba0814b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:36.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.394+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 shutdown_connections 2024-08-18T13:39:36.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.394+0000 7fba0d815700 1 -- 172.21.15.118:0/1095693356 wait complete. 2024-08-18T13:39:36.414 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-08-18T13:39:36.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.530+0000 7f13791da700 1 Processor -- start 2024-08-18T13:39:36.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.530+0000 7f13791da700 1 -- start start 2024-08-18T13:39:36.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f137406db00 0x7f1374158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f13740c3590 0x7f1374159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1374068c50 con 0x7f13740bec20 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137405eb10 con 0x7f13740c3590 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137406ab90 con 0x7f137406db00 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f13740c3590 0x7f1374159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f13740c3590 0x7f1374159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54072/0 (socket says 172.21.15.118:54072) 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 -- 172.21.15.118:0/3379146587 learned_addr learned my addr 172.21.15.118:0/3379146587 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:36.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13737fe700 1 --2- 172.21.15.118:0/3379146587 >> v2:172.21.15.178:3300/0 conn(0x7f137406db00 0x7f1374158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 -- 172.21.15.118:0/3379146587 >> v2:172.21.15.178:3300/0 conn(0x7f137406db00 msgr2=0x7f1374158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 --2- 172.21.15.118:0/3379146587 >> v2:172.21.15.178:3300/0 conn(0x7f137406db00 0x7f1374158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 -- 172.21.15.118:0/3379146587 >> v1:172.21.15.178:6789/0 conn(0x7f13740bec20 legacy=0x7f1374069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 -- 172.21.15.118:0/3379146587 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137406a8b0 con 0x7f13740c3590 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13789d9700 1 --2- 172.21.15.118:0/3379146587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13740c3590 0x7f1374159110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f137406ef80 tx=0x7f136c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c915890a255d19a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f1372ffd700 1 -- 172.21.15.118:0/3379146587 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00e070 con 0x7f13740c3590 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f1372ffd700 1 -- 172.21.15.118:0/3379146587 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c00ba50 con 0x7f13740c3590 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f1372ffd700 1 -- 172.21.15.118:0/3379146587 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0092e0 con 0x7f13740c3590 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- 172.21.15.118:0/3379146587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13740c3590 msgr2=0x7f1374159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 --2- 172.21.15.118:0/3379146587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13740c3590 0x7f1374159110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f137406ef80 tx=0x7f136c009840 comp rx=0 tx=0).stop 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- 172.21.15.118:0/3379146587 shutdown_connections 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 --2- 172.21.15.118:0/3379146587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13740c3590 0x7f1374159110 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 --2- 172.21.15.118:0/3379146587 >> v2:172.21.15.178:3300/0 conn(0x7f137406db00 0x7f1374158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- 172.21.15.118:0/3379146587 >> 172.21.15.118:0/3379146587 conn(0x7f137406c750 msgr2=0x7f13740c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:36.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- 172.21.15.118:0/3379146587 shutdown_connections 2024-08-18T13:39:36.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.534+0000 7f13791da700 1 -- 172.21.15.118:0/3379146587 wait complete. 2024-08-18T13:39:36.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 Processor -- start 2024-08-18T13:39:36.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 -- start start 2024-08-18T13:39:36.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f137406db00 0x7f1374165ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:36.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13740bec20 0x7f13740c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:36.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13740c3590 0x7f1374157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:36.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374152390 con 0x7f137406db00 2024-08-18T13:39:36.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1374146200 con 0x7f13740c3590 2024-08-18T13:39:36.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13791da700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740bfff0 con 0x7f13740bec20 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f137406db00 0x7f1374165ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f137406db00 0x7f1374165ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54084/0 (socket says 172.21.15.118:54084) 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 -- 172.21.15.118:0/77110489 learned_addr learned my addr 172.21.15.118:0/77110489 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13789d9700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13740c3590 0x7f1374157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f13737fe700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13740bec20 0x7f13740c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 -- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13740bec20 msgr2=0x7f13740c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13740bec20 0x7f13740c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 -- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13740c3590 msgr2=0x7f1374157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13740c3590 0x7f1374157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.538+0000 7f1373fff700 1 -- 172.21.15.118:0/77110489 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c009ae0 con 0x7f137406db00 2024-08-18T13:39:36.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f1373fff700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f137406db00 0x7f1374165ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c5c0 tx=0x7f135c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:36.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f136bfff700 1 -- 172.21.15.118:0/77110489 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c011590 con 0x7f137406db00 2024-08-18T13:39:36.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f136bfff700 1 -- 172.21.15.118:0/77110489 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c011710 con 0x7f137406db00 2024-08-18T13:39:36.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1374146450 con 0x7f137406db00 2024-08-18T13:39:36.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f136bfff700 1 -- 172.21.15.118:0/77110489 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c011ad0 con 0x7f137406db00 2024-08-18T13:39:36.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13740bf9a0 con 0x7f137406db00 2024-08-18T13:39:36.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374071410 con 0x7f137406db00 2024-08-18T13:39:36.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f136bfff700 1 -- 172.21.15.118:0/77110489 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f135c01b3b0 con 0x7f137406db00 2024-08-18T13:39:36.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f136bfff700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1354060860 0x7f1354062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:36.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f136bfff700 1 -- 172.21.15.118:0/77110489 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5931+0+0 (secure 0 0 0) 0x7f135c07c0d0 con 0x7f137406db00 2024-08-18T13:39:36.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.542+0000 7f13737fe700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1354060860 0x7f1354062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:36.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.546+0000 7f13737fe700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1354060860 0x7f1354062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13740c1ca0 tx=0x7f1364008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:36.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.550+0000 7f136bfff700 1 -- 172.21.15.118:0/77110489 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f135c04c440 con 0x7f137406db00 2024-08-18T13:39:36.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:36.718+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1374152d00 con 0x7f137406db00 2024-08-18T13:39:38.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.406+0000 7f136bfff700 1 -- 172.21.15.118:0/77110489 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f135c04fa60 con 0x7f137406db00 2024-08-18T13:39:38.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1354060860 msgr2=0x7f1354062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:38.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1354060860 0x7f1354062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13740c1ca0 tx=0x7f1364008040 comp rx=0 tx=0).stop 2024-08-18T13:39:38.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f137406db00 msgr2=0x7f1374165ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:38.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f137406db00 0x7f1374165ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c5c0 tx=0x7f135c00c8d0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 shutdown_connections 2024-08-18T13:39:38.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1354060860 0x7f1354062d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13740c3590 0x7f1374157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13740bec20 0x7f13740c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 --2- 172.21.15.118:0/77110489 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f137406db00 0x7f1374165ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 >> 172.21.15.118:0/77110489 conn(0x7f137406c750 msgr2=0x7f137405df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:38.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 shutdown_connections 2024-08-18T13:39:38.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.422+0000 7f13791da700 1 -- 172.21.15.118:0/77110489 wait complete. 2024-08-18T13:39:38.429 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_data_ec' created 2024-08-18T13:39:38.440 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-08-18T13:39:38.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.546+0000 7fb80d06e700 1 Processor -- start 2024-08-18T13:39:38.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.546+0000 7fb80d06e700 1 -- start start 2024-08-18T13:39:38.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c0cf0 0x7fb808155d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:38.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb8080cc430 0x7fb80814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:38.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8080698b0 con 0x7fb8080c10c0 2024-08-18T13:39:38.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb808156250 con 0x7fb8080cc430 2024-08-18T13:39:38.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806a450 con 0x7fb8080c0cf0 2024-08-18T13:39:38.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb806d9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c0cf0 0x7fb808155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:38.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80759e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb8080cc430 0x7fb80814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:38.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb806d9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c0cf0 0x7fb808155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:38194/0 (socket says 172.21.15.118:38194) 2024-08-18T13:39:38.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb806d9d700 1 -- 172.21.15.118:0/679804242 learned_addr learned my addr 172.21.15.118:0/679804242 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:38.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80759e700 1 -- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c0cf0 msgr2=0x7fb808155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:38.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80759e700 1 --2- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c0cf0 0x7fb808155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80759e700 1 -- 172.21.15.118:0/679804242 >> v1:172.21.15.178:6789/0 conn(0x7fb8080c10c0 legacy=0x7fb808069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:38.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80759e700 1 -- 172.21.15.118:0/679804242 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080ccb70 con 0x7fb8080cc430 2024-08-18T13:39:38.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb806d9d700 1 --2- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c0cf0 0x7fb808155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:38.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80759e700 1 --2- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080cc430 0x7fb80814efc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb800009e60 tx=0x7fb800009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb16d26dc249eae5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb805d9b700 1 -- 172.21.15.118:0/679804242 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80000e070 con 0x7fb8080cc430 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb805d9b700 1 -- 172.21.15.118:0/679804242 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80000bbf0 con 0x7fb8080cc430 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb805d9b700 1 -- 172.21.15.118:0/679804242 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800012430 con 0x7fb8080cc430 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080cc430 msgr2=0x7fb80814efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 --2- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080cc430 0x7fb80814efc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb800009e60 tx=0x7fb800009780 comp rx=0 tx=0).stop 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- 172.21.15.118:0/679804242 shutdown_connections 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 --2- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080cc430 0x7fb80814efc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 --2- 172.21.15.118:0/679804242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c0cf0 0x7fb808155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- 172.21.15.118:0/679804242 >> 172.21.15.118:0/679804242 conn(0x7fb808067df0 msgr2=0x7fb8080cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:38.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- 172.21.15.118:0/679804242 shutdown_connections 2024-08-18T13:39:38.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.550+0000 7fb80d06e700 1 -- 172.21.15.118:0/679804242 wait complete. 2024-08-18T13:39:38.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 Processor -- start 2024-08-18T13:39:38.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 -- start start 2024-08-18T13:39:38.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080c0cf0 0x7fb8080c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:38.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c10c0 0x7fb808153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8080cc430 0x7fb8081540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb808154610 con 0x7fb8080cc430 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081586e0 con 0x7fb8080c0cf0 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80d06e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081580d0 con 0x7fb8080c10c0 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb806d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080c0cf0 0x7fb8080c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8080cc430 0x7fb8081540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80659c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c10c0 0x7fb808153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8080cc430 0x7fb8081540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54100/0 (socket says 172.21.15.118:54100) 2024-08-18T13:39:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.554+0000 7fb80759e700 1 -- 172.21.15.118:0/3806046281 learned_addr learned my addr 172.21.15.118:0/3806046281 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:38.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80759e700 1 -- 172.21.15.118:0/3806046281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c10c0 msgr2=0x7fb808153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:38.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80759e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c10c0 0x7fb808153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80759e700 1 -- 172.21.15.118:0/3806046281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080c0cf0 msgr2=0x7fb8080c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80759e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080c0cf0 0x7fb8080c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80759e700 1 -- 172.21.15.118:0/3806046281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80094f0 con 0x7fb8080cc430 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80759e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8080cc430 0x7fb8081540d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb80000b6c0 tx=0x7fb800019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb7feffd700 1 -- 172.21.15.118:0/3806046281 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800012e20 con 0x7fb8080cc430 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb800009a20 con 0x7fb8080cc430 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb7feffd700 1 -- 172.21.15.118:0/3806046281 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80001d630 con 0x7fb8080cc430 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb7feffd700 1 -- 172.21.15.118:0/3806046281 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80001d9f0 con 0x7fb8080cc430 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb808157330 con 0x7fb8080cc430 2024-08-18T13:39:38.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb808153160 con 0x7fb8080cc430 2024-08-18T13:39:38.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.558+0000 7fb7feffd700 1 -- 172.21.15.118:0/3806046281 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb800024070 con 0x7fb8080cc430 2024-08-18T13:39:38.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.562+0000 7fb7feffd700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e8060890 0x7fb7e8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:38.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.562+0000 7fb7feffd700 1 -- 172.21.15.118:0/3806046281 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb80007f450 con 0x7fb8080cc430 2024-08-18T13:39:38.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.562+0000 7fb806d9d700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e8060890 0x7fb7e8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:38.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.562+0000 7fb806d9d700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e8060890 0x7fb7e8062d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8006d20 tx=0x7fb7f8009250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:38.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.566+0000 7fb7feffd700 1 -- 172.21.15.118:0/3806046281 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb80004f5d0 con 0x7fb8080cc430 2024-08-18T13:39:38.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:38.730+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb808153160 con 0x7fb8080cc430 2024-08-18T13:39:39.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.414+0000 7fb7feffd700 1 -- 172.21.15.118:0/3806046281 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb808153160 con 0x7fb8080cc430 2024-08-18T13:39:39.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e8060890 msgr2=0x7fb7e8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:39.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e8060890 0x7fb7e8062d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8006d20 tx=0x7fb7f8009250 comp rx=0 tx=0).stop 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8080cc430 msgr2=0x7fb8081540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8080cc430 0x7fb8081540d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb80000b6c0 tx=0x7fb800019000 comp rx=0 tx=0).stop 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 shutdown_connections 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e8060890 0x7fb7e8062d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb8080cc430 0x7fb8081540d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8080c10c0 0x7fb808153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 --2- 172.21.15.118:0/3806046281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8080c0cf0 0x7fb8080c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 >> 172.21.15.118:0/3806046281 conn(0x7fb808067df0 msgr2=0x7fb80814e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 shutdown_connections 2024-08-18T13:39:39.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.430+0000 7fb80d06e700 1 -- 172.21.15.118:0/3806046281 wait complete. 2024-08-18T13:39:39.438 INFO:teuthology.orchestra.run.smithi118.stderr:set pool 4 allow_ec_overwrites to true 2024-08-18T13:39:39.460 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-08-18T13:39:39.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.558+0000 7f156b59e700 1 Processor -- start 2024-08-18T13:39:39.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.558+0000 7f156b59e700 1 -- start start 2024-08-18T13:39:39.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c0c4620 0x7f156c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:39.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f156c154bd0 0x7f156c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:39.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0bea10 con 0x7f156c14d220 2024-08-18T13:39:39.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0bef40 con 0x7f156c154bd0 2024-08-18T13:39:39.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0c0480 con 0x7f156c0c4620 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f156c154bd0 0x7f156c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f156c154bd0 0x7f156c156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54104/0 (socket says 172.21.15.118:54104) 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 -- 172.21.15.118:0/512494297 learned_addr learned my addr 172.21.15.118:0/512494297 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f1561d9b700 1 --2- 172.21.15.118:0/512494297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c0c4620 0x7f156c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 -- 172.21.15.118:0/512494297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c0c4620 msgr2=0x7f156c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 --2- 172.21.15.118:0/512494297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c0c4620 0x7f156c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 -- 172.21.15.118:0/512494297 >> v1:172.21.15.178:6789/0 conn(0x7f156c14d220 legacy=0x7f156c1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 -- 172.21.15.118:0/512494297 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0c0760 con 0x7f156c154bd0 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156ad9d700 1 --2- 172.21.15.118:0/512494297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c154bd0 0x7f156c156b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1564009f20 tx=0x7f1564009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9f9e1112076a1f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f1569d9b700 1 -- 172.21.15.118:0/512494297 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400e070 con 0x7f156c154bd0 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f1569d9b700 1 -- 172.21.15.118:0/512494297 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156400bb10 con 0x7f156c154bd0 2024-08-18T13:39:39.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f1569d9b700 1 -- 172.21.15.118:0/512494297 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640092e0 con 0x7f156c154bd0 2024-08-18T13:39:39.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- 172.21.15.118:0/512494297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c154bd0 msgr2=0x7f156c156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:39.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 --2- 172.21.15.118:0/512494297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c154bd0 0x7f156c156b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1564009f20 tx=0x7f1564009840 comp rx=0 tx=0).stop 2024-08-18T13:39:39.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- 172.21.15.118:0/512494297 shutdown_connections 2024-08-18T13:39:39.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 --2- 172.21.15.118:0/512494297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c154bd0 0x7f156c156b00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 --2- 172.21.15.118:0/512494297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c0c4620 0x7f156c1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- 172.21.15.118:0/512494297 >> 172.21.15.118:0/512494297 conn(0x7f156c06d5a0 msgr2=0x7f156c06d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:39.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- 172.21.15.118:0/512494297 shutdown_connections 2024-08-18T13:39:39.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.562+0000 7f156b59e700 1 -- 172.21.15.118:0/512494297 wait complete. 2024-08-18T13:39:39.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 Processor -- start 2024-08-18T13:39:39.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 -- start start 2024-08-18T13:39:39.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c14d220 0x7f156c162f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:39.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c154bd0 0x7f156c14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c14e2b0 0x7f156c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c14ec80 con 0x7f156c14e2b0 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c14f740 con 0x7f156c14d220 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c14d860 con 0x7f156c154bd0 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c14e2b0 0x7f156c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f1561d9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c154bd0 0x7f156c14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c14e2b0 0x7f156c14b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54116/0 (socket says 172.21.15.118:54116) 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 -- 172.21.15.118:0/1019001157 learned_addr learned my addr 172.21.15.118:0/1019001157 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:39.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156a59c700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c14d220 0x7f156c162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:39.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 -- 172.21.15.118:0/1019001157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c154bd0 msgr2=0x7f156c14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:39.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c154bd0 0x7f156c14a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 -- 172.21.15.118:0/1019001157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c14d220 msgr2=0x7f156c162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:39.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c14d220 0x7f156c162f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:39.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 -- 172.21.15.118:0/1019001157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564009ae0 con 0x7f156c14e2b0 2024-08-18T13:39:39.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156ad9d700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c14e2b0 0x7f156c14b800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f156c070bb0 tx=0x7f1564018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:39.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f1562ffd700 1 -- 172.21.15.118:0/1019001157 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564008ce0 con 0x7f156c14e2b0 2024-08-18T13:39:39.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f1562ffd700 1 -- 172.21.15.118:0/1019001157 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1564008e40 con 0x7f156c14e2b0 2024-08-18T13:39:39.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c14de20 con 0x7f156c14e2b0 2024-08-18T13:39:39.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f1562ffd700 1 -- 172.21.15.118:0/1019001157 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156401d670 con 0x7f156c14e2b0 2024-08-18T13:39:39.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.566+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c150460 con 0x7f156c14e2b0 2024-08-18T13:39:39.575 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.570+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c14dfb0 con 0x7f156c14e2b0 2024-08-18T13:39:39.575 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.570+0000 7f1562ffd700 1 -- 172.21.15.118:0/1019001157 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1564003040 con 0x7f156c14e2b0 2024-08-18T13:39:39.576 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.570+0000 7f1562ffd700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f154c060890 0x7f154c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:39.576 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.570+0000 7f1562ffd700 1 -- 172.21.15.118:0/1019001157 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f156407ea30 con 0x7f156c14e2b0 2024-08-18T13:39:39.576 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.570+0000 7f156a59c700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f154c060890 0x7f154c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:39.577 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.574+0000 7f156a59c700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f154c060890 0x7f154c062d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f155c003a80 tx=0x7f155c007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:39.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.582+0000 7f1562ffd700 1 -- 172.21.15.118:0/1019001157 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f156404ebb0 con 0x7f156c14e2b0 2024-08-18T13:39:39.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:39.782+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f156c071410 con 0x7f156c14e2b0 2024-08-18T13:39:40.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.446+0000 7f1562ffd700 1 -- 172.21.15.118:0/1019001157 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) v1 ==== 137+0+0 (secure 0 0 0) 0x7f15640521d0 con 0x7f156c14e2b0 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f154c060890 msgr2=0x7f154c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f154c060890 0x7f154c062d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f155c003a80 tx=0x7f155c007ec0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c14e2b0 msgr2=0x7f156c14b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c14e2b0 0x7f156c14b800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f156c070bb0 tx=0x7f1564018000 comp rx=0 tx=0).stop 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 shutdown_connections 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f154c060890 0x7f154c062d40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156c14e2b0 0x7f156c14b800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156c154bd0 0x7f156c14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 --2- 172.21.15.118:0/1019001157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c14d220 0x7f156c162f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 >> 172.21.15.118:0/1019001157 conn(0x7f156c06d5a0 msgr2=0x7f156c0c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:40.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 shutdown_connections 2024-08-18T13:39:40.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.450+0000 7f156b59e700 1 -- 172.21.15.118:0/1019001157 wait complete. 2024-08-18T13:39:40.457 INFO:teuthology.orchestra.run.smithi118.stderr:added data pool 4 to fsmap 2024-08-18T13:39:40.471 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:39:40.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35f59e700 1 Processor -- start 2024-08-18T13:39:40.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35f59e700 1 -- start start 2024-08-18T13:39:40.580 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35f59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3600aeba0 0x7fd360155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.580 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abaf0 0x7fd3600c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.580 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35f59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3600b0ee0 con 0x7fd3600abef0 2024-08-18T13:39:40.580 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35f59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3600b1180 con 0x7fd3600aeba0 2024-08-18T13:39:40.580 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600b1420 con 0x7fd3600abaf0 2024-08-18T13:39:40.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35dd9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abaf0 0x7fd3600c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35e59c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3600aeba0 0x7fd360155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35dd9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abaf0 0x7fd3600c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:38220/0 (socket says 172.21.15.118:38220) 2024-08-18T13:39:40.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35e59c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3600aeba0 0x7fd360155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:39824/0 (socket says 172.21.15.118:39824) 2024-08-18T13:39:40.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35dd9b700 1 -- 172.21.15.118:0/1763733911 learned_addr learned my addr 172.21.15.118:0/1763733911 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:40.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35dd9b700 1 -- 172.21.15.118:0/1763733911 >> v2:172.21.15.178:3300/0 conn(0x7fd3600aeba0 msgr2=0x7fd360155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:40.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35dd9b700 1 --2- 172.21.15.118:0/1763733911 >> v2:172.21.15.178:3300/0 conn(0x7fd3600aeba0 0x7fd360155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35dd9b700 1 -- 172.21.15.118:0/1763733911 >> v1:172.21.15.178:6789/0 conn(0x7fd3600abef0 legacy=0x7fd360158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.574+0000 7fd35dd9b700 1 -- 172.21.15.118:0/1763733911 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600b16c0 con 0x7fd3600abaf0 2024-08-18T13:39:40.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35dd9b700 1 --2- 172.21.15.118:0/1763733911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abaf0 0x7fd3600c6a10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd3500099b0 tx=0x7fd350009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a93bc11e7fafa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:40.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35d59a700 1 -- 172.21.15.118:0/1763733911 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350009ea0 con 0x7fd3600abaf0 2024-08-18T13:39:40.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35d59a700 1 -- 172.21.15.118:0/1763733911 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35000b8f0 con 0x7fd3600abaf0 2024-08-18T13:39:40.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35d59a700 1 -- 172.21.15.118:0/1763733911 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35000bc90 con 0x7fd3600abaf0 2024-08-18T13:39:40.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 -- 172.21.15.118:0/1763733911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abaf0 msgr2=0x7fd3600c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 --2- 172.21.15.118:0/1763733911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abaf0 0x7fd3600c6a10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd3500099b0 tx=0x7fd350009250 comp rx=0 tx=0).stop 2024-08-18T13:39:40.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 -- 172.21.15.118:0/1763733911 shutdown_connections 2024-08-18T13:39:40.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 --2- 172.21.15.118:0/1763733911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abaf0 0x7fd3600c6a10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 --2- 172.21.15.118:0/1763733911 >> v2:172.21.15.178:3300/0 conn(0x7fd3600aeba0 0x7fd360155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 -- 172.21.15.118:0/1763733911 >> 172.21.15.118:0/1763733911 conn(0x7fd3600cbb80 msgr2=0x7fd3600cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:40.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 -- 172.21.15.118:0/1763733911 shutdown_connections 2024-08-18T13:39:40.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 -- 172.21.15.118:0/1763733911 wait complete. 2024-08-18T13:39:40.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 Processor -- start 2024-08-18T13:39:40.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.578+0000 7fd35f59e700 1 -- start start 2024-08-18T13:39:40.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abef0 0x7fd3600c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3600aeba0 0x7fd36014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3601652f0 0x7fd3601687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c4c80 con 0x7fd3601652f0 2024-08-18T13:39:40.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c3e10 con 0x7fd3600aeba0 2024-08-18T13:39:40.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c4070 con 0x7fd3600abef0 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3601652f0 0x7fd3601687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3601652f0 0x7fd3601687f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54124/0 (socket says 172.21.15.118:54124) 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35dd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3600aeba0 0x7fd36014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 -- 172.21.15.118:0/2637316645 learned_addr learned my addr 172.21.15.118:0/2637316645 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35e59c700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abef0 0x7fd3600c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 -- 172.21.15.118:0/2637316645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abef0 msgr2=0x7fd3600c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abef0 0x7fd3600c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 -- 172.21.15.118:0/2637316645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3600aeba0 msgr2=0x7fd36014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3600aeba0 0x7fd36014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 -- 172.21.15.118:0/2637316645 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3500094f0 con 0x7fd3601652f0 2024-08-18T13:39:40.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35ed9d700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3601652f0 0x7fd3601687f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd35800c8e0 tx=0x7fd358001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:40.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd3567fc700 1 -- 172.21.15.118:0/2637316645 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358012630 con 0x7fd3601652f0 2024-08-18T13:39:40.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd3567fc700 1 -- 172.21.15.118:0/2637316645 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd358012790 con 0x7fd3601652f0 2024-08-18T13:39:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd3567fc700 1 -- 172.21.15.118:0/2637316645 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358012b50 con 0x7fd3601652f0 2024-08-18T13:39:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3600c4350 con 0x7fd3601652f0 2024-08-18T13:39:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.582+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3600c4910 con 0x7fd3601652f0 2024-08-18T13:39:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.586+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3600c4df0 con 0x7fd3601652f0 2024-08-18T13:39:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.586+0000 7fd3567fc700 1 -- 172.21.15.118:0/2637316645 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd35801a3a0 con 0x7fd3601652f0 2024-08-18T13:39:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.586+0000 7fd3567fc700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd340060860 0x7fd340062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.586+0000 7fd3567fc700 1 -- 172.21.15.118:0/2637316645 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fd35807d0b0 con 0x7fd3601652f0 2024-08-18T13:39:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.586+0000 7fd35e59c700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd340060860 0x7fd340062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.586+0000 7fd35e59c700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd340060860 0x7fd340062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d10 tx=0x7fd3480091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:40.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.594+0000 7fd3567fc700 1 -- 172.21.15.118:0/2637316645 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd358050ca0 con 0x7fd3601652f0 2024-08-18T13:39:40.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.786+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3600cb460 con 0x7fd3601652f0 2024-08-18T13:39:40.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd3567fc700 1 -- 172.21.15.118:0/2637316645 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+6123 (secure 0 0 0) 0x7fd358050830 con 0x7fd3601652f0 2024-08-18T13:39:40.793 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:40.794 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:40.441441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-08-18T13:39:40.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd340060860 msgr2=0x7fd340062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd340060860 0x7fd340062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d10 tx=0x7fd3480091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3601652f0 msgr2=0x7fd3601687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3601652f0 0x7fd3601687f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd35800c8e0 tx=0x7fd358001710 comp rx=0 tx=0).stop 2024-08-18T13:39:40.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 shutdown_connections 2024-08-18T13:39:40.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd340060860 0x7fd340062d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd3601652f0 0x7fd3601687f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3600aeba0 0x7fd36014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 --2- 172.21.15.118:0/2637316645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3600abef0 0x7fd3600c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 >> 172.21.15.118:0/2637316645 conn(0x7fd3600cbb80 msgr2=0x7fd3600c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:40.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 shutdown_connections 2024-08-18T13:39:40.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.790+0000 7fd35f59e700 1 -- 172.21.15.118:0/2637316645 wait complete. 2024-08-18T13:39:40.797 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 6 2024-08-18T13:39:40.807 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:40.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.906+0000 7f7456afe700 1 Processor -- start 2024-08-18T13:39:40.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.906+0000 7f7456afe700 1 -- start start 2024-08-18T13:39:40.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.906+0000 7f7456afe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74500c46f0 0x7f7450156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.906+0000 7f7456afe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7450067d50 0x7f7450153980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.906+0000 7f7456afe700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74501534f0 con 0x7f74500c4320 2024-08-18T13:39:40.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.906+0000 7f7456afe700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74501574c0 con 0x7f7450067d50 2024-08-18T13:39:40.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.906+0000 7f7456afe700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7450152f70 con 0x7f74500c46f0 2024-08-18T13:39:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7450067d50 0x7f7450153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7450067d50 0x7f7450153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54130/0 (socket says 172.21.15.118:54130) 2024-08-18T13:39:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74552fb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74500c46f0 0x7f7450156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 -- 172.21.15.118:0/3841020979 learned_addr learned my addr 172.21.15.118:0/3841020979 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7454afa700 1 -- 172.21.15.118:0/3841020979 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114503434 0 0) 0x7f74501534f0 con 0x7f74500c4320 2024-08-18T13:39:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 -- 172.21.15.118:0/3841020979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74500c46f0 msgr2=0x7f7450156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 --2- 172.21.15.118:0/3841020979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74500c46f0 0x7f7450156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 -- 172.21.15.118:0/3841020979 >> v1:172.21.15.178:6789/0 conn(0x7f74500c4320 legacy=0x7f745005e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 -- 172.21.15.118:0/3841020979 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450156c20 con 0x7f7450067d50 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f74562fd700 1 --2- 172.21.15.118:0/3841020979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7450067d50 0x7f7450153980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74400099b0 tx=0x7f7440009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d93d5d29328e31d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7454afa700 1 -- 172.21.15.118:0/3841020979 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440009ea0 con 0x7f7450067d50 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7454afa700 1 -- 172.21.15.118:0/3841020979 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744000b8f0 con 0x7f7450067d50 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7454afa700 1 -- 172.21.15.118:0/3841020979 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000bc90 con 0x7f7450067d50 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 -- 172.21.15.118:0/3841020979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7450067d50 msgr2=0x7f7450153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 --2- 172.21.15.118:0/3841020979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7450067d50 0x7f7450153980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74400099b0 tx=0x7f7440009250 comp rx=0 tx=0).stop 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 -- 172.21.15.118:0/3841020979 shutdown_connections 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 --2- 172.21.15.118:0/3841020979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7450067d50 0x7f7450153980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 --2- 172.21.15.118:0/3841020979 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74500c46f0 0x7f7450156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 -- 172.21.15.118:0/3841020979 >> 172.21.15.118:0/3841020979 conn(0x7f74500c81e0 msgr2=0x7f74500c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:40.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 -- 172.21.15.118:0/3841020979 shutdown_connections 2024-08-18T13:39:40.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.910+0000 7f7456afe700 1 -- 172.21.15.118:0/3841020979 wait complete. 2024-08-18T13:39:40.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 Processor -- start 2024-08-18T13:39:40.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 -- start start 2024-08-18T13:39:40.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7450067d50 0x7f7450155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74500c4320 0x7f74500c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74500c46f0 0x7f74500c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500b6f50 con 0x7f74500c4320 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7450156870 con 0x7f74500c46f0 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7450168bd0 con 0x7f7450067d50 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74500c4320 0x7f74500c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74500c4320 0x7f74500c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54144/0 (socket says 172.21.15.118:54144) 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 -- 172.21.15.118:0/131205084 learned_addr learned my addr 172.21.15.118:0/131205084 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7455afc700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7450067d50 0x7f7450155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74562fd700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74500c46f0 0x7f74500c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 -- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7450067d50 msgr2=0x7f7450155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7450067d50 0x7f7450155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 -- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74500c46f0 msgr2=0x7f74500c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74500c46f0 0x7f74500c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 -- 172.21.15.118:0/131205084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74400094f0 con 0x7f74500c4320 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74562fd700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74500c46f0 0x7f74500c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f74552fb700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74500c4320 0x7f74500c7390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7438009e50 tx=0x7f743800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f744dffb700 1 -- 172.21.15.118:0/131205084 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743800cbe0 con 0x7f74500c4320 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f744dffb700 1 -- 172.21.15.118:0/131205084 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743800f040 con 0x7f74500c4320 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7450168000 con 0x7f74500c4320 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f744dffb700 1 -- 172.21.15.118:0/131205084 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438014480 con 0x7f74500c4320 2024-08-18T13:39:40.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.914+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74501682a0 con 0x7f74500c4320 2024-08-18T13:39:40.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.918+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74500c1d00 con 0x7f74500c4320 2024-08-18T13:39:40.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.918+0000 7f744dffb700 1 -- 172.21.15.118:0/131205084 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f74380147f0 con 0x7f74500c4320 2024-08-18T13:39:40.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.918+0000 7f744dffb700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7430060860 0x7f7430062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:40.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.918+0000 7f744dffb700 1 -- 172.21.15.118:0/131205084 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f743807d820 con 0x7f74500c4320 2024-08-18T13:39:40.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.918+0000 7f7455afc700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7430060860 0x7f7430062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:40.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.918+0000 7f7455afc700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7430060860 0x7f7430062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7448007fd0 tx=0x7f7448007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:40.931 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:40.926+0000 7f744dffb700 1 -- 172.21.15.118:0/131205084 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f74500c1d00 con 0x7f74500c4320 2024-08-18T13:39:41.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.090+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74500c1d00 con 0x7f74500c4320 2024-08-18T13:39:41.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f744dffb700 1 -- 172.21.15.118:0/131205084 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f74500c1d00 con 0x7f74500c4320 2024-08-18T13:39:41.098 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:41.099 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7430060860 msgr2=0x7f7430062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7430060860 0x7f7430062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7448007fd0 tx=0x7f7448007d30 comp rx=0 tx=0).stop 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74500c4320 msgr2=0x7f74500c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74500c4320 0x7f74500c7390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7438009e50 tx=0x7f743800b040 comp rx=0 tx=0).stop 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 shutdown_connections 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7430060860 0x7f7430062d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74500c46f0 0x7f74500c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74500c4320 0x7f74500c7390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 --2- 172.21.15.118:0/131205084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7450067d50 0x7f7450155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 >> 172.21.15.118:0/131205084 conn(0x7f74500c81e0 msgr2=0x7f7450068230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:41.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 shutdown_connections 2024-08-18T13:39:41.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.094+0000 7f7456afe700 1 -- 172.21.15.118:0/131205084 wait complete. 2024-08-18T13:39:41.113 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:41.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51dc932700 1 Processor -- start 2024-08-18T13:39:41.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51dc932700 1 -- start start 2024-08-18T13:39:41.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51dc932700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f51d806b620 0x7f51d806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51dc932700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c4030 0x7f51d8152c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51dc932700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d806bf30 con 0x7f51d80c2630 2024-08-18T13:39:41.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51dc932700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d8068ac0 con 0x7f51d806b620 2024-08-18T13:39:41.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51dc932700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8153160 con 0x7f51d80c4030 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51cffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c4030 0x7f51d8152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51d659c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f51d806b620 0x7f51d806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51cffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c4030 0x7f51d8152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:38238/0 (socket says 172.21.15.118:38238) 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51cffff700 1 -- 172.21.15.118:0/737357247 learned_addr learned my addr 172.21.15.118:0/737357247 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51cffff700 1 -- 172.21.15.118:0/737357247 >> v2:172.21.15.178:3300/0 conn(0x7f51d806b620 msgr2=0x7f51d806b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51cffff700 1 --2- 172.21.15.118:0/737357247 >> v2:172.21.15.178:3300/0 conn(0x7f51d806b620 0x7f51d806b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51cffff700 1 -- 172.21.15.118:0/737357247 >> v1:172.21.15.178:6789/0 conn(0x7f51d80c2630 legacy=0x7f51d814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.214+0000 7f51cffff700 1 -- 172.21.15.118:0/737357247 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8068580 con 0x7f51d80c4030 2024-08-18T13:39:41.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51cffff700 1 --2- 172.21.15.118:0/737357247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c4030 0x7f51d8152c20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51c00099b0 tx=0x7f51c0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f26863517d1421b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51d5d9b700 1 -- 172.21.15.118:0/737357247 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0009ea0 con 0x7f51d80c4030 2024-08-18T13:39:41.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51d5d9b700 1 -- 172.21.15.118:0/737357247 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000b860 con 0x7f51d80c4030 2024-08-18T13:39:41.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51d5d9b700 1 -- 172.21.15.118:0/737357247 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000bc00 con 0x7f51d80c4030 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- 172.21.15.118:0/737357247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c4030 msgr2=0x7f51d8152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 --2- 172.21.15.118:0/737357247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c4030 0x7f51d8152c20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51c00099b0 tx=0x7f51c0009250 comp rx=0 tx=0).stop 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- 172.21.15.118:0/737357247 shutdown_connections 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 --2- 172.21.15.118:0/737357247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c4030 0x7f51d8152c20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 --2- 172.21.15.118:0/737357247 >> v2:172.21.15.178:3300/0 conn(0x7f51d806b620 0x7f51d806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- 172.21.15.118:0/737357247 >> 172.21.15.118:0/737357247 conn(0x7f51d80c7c40 msgr2=0x7f51d80c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- 172.21.15.118:0/737357247 shutdown_connections 2024-08-18T13:39:41.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- 172.21.15.118:0/737357247 wait complete. 2024-08-18T13:39:41.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 Processor -- start 2024-08-18T13:39:41.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- start start 2024-08-18T13:39:41.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51d806b620 0x7f51d8154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c2630 0x7f51d8149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51d81599c0 0x7f51d81555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80c5bd0 con 0x7f51d81599c0 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8154e90 con 0x7f51d806b620 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.218+0000 7f51dc932700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8156860 con 0x7f51d80c2630 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51d81599c0 0x7f51d81555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51d81599c0 0x7f51d81555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54160/0 (socket says 172.21.15.118:54160) 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 -- 172.21.15.118:0/3610064014 learned_addr learned my addr 172.21.15.118:0/3610064014 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d659c700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51d806b620 0x7f51d8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51cffff700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c2630 0x7f51d8149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 -- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c2630 msgr2=0x7f51d8149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c2630 0x7f51d8149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 -- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51d806b620 msgr2=0x7f51d8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51d806b620 0x7f51d8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 -- 172.21.15.118:0/3610064014 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00094f0 con 0x7f51d81599c0 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d659c700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51d806b620 0x7f51d8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:41.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51cffff700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c2630 0x7f51d8149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:41.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51d6d9d700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51d81599c0 0x7f51d81555d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c910 tx=0x7f51d0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51ce7fc700 1 -- 172.21.15.118:0/3610064014 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000a740 con 0x7f51d81599c0 2024-08-18T13:39:41.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51ce7fc700 1 -- 172.21.15.118:0/3610064014 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d000a8a0 con 0x7f51d81599c0 2024-08-18T13:39:41.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51ce7fc700 1 -- 172.21.15.118:0/3610064014 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000ac60 con 0x7f51d81599c0 2024-08-18T13:39:41.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d814f4a0 con 0x7f51d81599c0 2024-08-18T13:39:41.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d814f740 con 0x7f51d81599c0 2024-08-18T13:39:41.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.222+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d80642f0 con 0x7f51d81599c0 2024-08-18T13:39:41.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.226+0000 7f51ce7fc700 1 -- 172.21.15.118:0/3610064014 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f51d0012420 con 0x7f51d81599c0 2024-08-18T13:39:41.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.226+0000 7f51ce7fc700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f51bc060890 0x7f51bc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.226+0000 7f51ce7fc700 1 -- 172.21.15.118:0/3610064014 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f51d007d110 con 0x7f51d81599c0 2024-08-18T13:39:41.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.226+0000 7f51d659c700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f51bc060890 0x7f51bc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.226+0000 7f51d659c700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f51bc060890 0x7f51bc062d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009d80 tx=0x7f51c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.234+0000 7f51ce7fc700 1 -- 172.21.15.118:0/3610064014 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f51d0050d00 con 0x7f51d81599c0 2024-08-18T13:39:41.322 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:39:41.326 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:39:41.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.394+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51d80c4510 con 0x7f51d81599c0 2024-08-18T13:39:41.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.394+0000 7f51ce7fc700 1 -- 172.21.15.118:0/3610064014 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f51d0050890 con 0x7f51d81599c0 2024-08-18T13:39:41.401 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:41.401 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:41.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f51bc060890 msgr2=0x7f51bc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f51bc060890 0x7f51bc062d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009d80 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2024-08-18T13:39:41.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51d81599c0 msgr2=0x7f51d81555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51d81599c0 0x7f51d81555d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c910 tx=0x7f51d0001710 comp rx=0 tx=0).stop 2024-08-18T13:39:41.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 shutdown_connections 2024-08-18T13:39:41.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f51bc060890 0x7f51bc062d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51d81599c0 0x7f51d81555d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d80c2630 0x7f51d8149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 --2- 172.21.15.118:0/3610064014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51d806b620 0x7f51d8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 >> 172.21.15.118:0/3610064014 conn(0x7f51d80c7c40 msgr2=0x7f51d80673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:41.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 shutdown_connections 2024-08-18T13:39:41.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.398+0000 7f51dc932700 1 -- 172.21.15.118:0/3610064014 wait complete. 2024-08-18T13:39:41.432 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:39:41.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506227e700 1 Processor -- start 2024-08-18T13:39:41.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506227e700 1 -- start start 2024-08-18T13:39:41.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506227e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c14f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506227e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f505c0897f0 0x7f505c14f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506227e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c068d50 con 0x7f505c0c0cf0 2024-08-18T13:39:41.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506227e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0c2270 con 0x7f505c0897f0 2024-08-18T13:39:41.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506227e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0698b0 con 0x7f505c068260 2024-08-18T13:39:41.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505b7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505bfff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f505c0c0cf0 0x7f505c069d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:56742/0 (socket says 172.21.15.118:56742) 2024-08-18T13:39:41.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505bfff700 1 -- 172.21.15.118:0/1047820533 learned_addr learned my addr 172.21.15.118:0/1047820533 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f506081b700 1 --2- 172.21.15.118:0/1047820533 >> v2:172.21.15.178:3300/0 conn(0x7f505c0897f0 0x7f505c14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505affd700 1 -- 172.21.15.118:0/1047820533 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215892471 0 0) 0x7f505c068d50 con 0x7f505c0c0cf0 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505affd700 1 -- 172.21.15.118:0/1047820533 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c003570 con 0x7f505c0c0cf0 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505b7fe700 1 -- 172.21.15.118:0/1047820533 >> v2:172.21.15.178:3300/0 conn(0x7f505c0897f0 msgr2=0x7f505c14f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505b7fe700 1 --2- 172.21.15.118:0/1047820533 >> v2:172.21.15.178:3300/0 conn(0x7f505c0897f0 0x7f505c14f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505b7fe700 1 -- 172.21.15.118:0/1047820533 >> v1:172.21.15.178:6789/0 conn(0x7f505c0c0cf0 legacy=0x7f505c069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.538+0000 7f505b7fe700 1 -- 172.21.15.118:0/1047820533 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c156090 con 0x7f505c068260 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f505b7fe700 1 --2- 172.21.15.118:0/1047820533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c14f130 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f50440098e0 tx=0x7f50440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de414101a3407a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f505affd700 1 -- 172.21.15.118:0/1047820533 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044011630 con 0x7f505c068260 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f505affd700 1 -- 172.21.15.118:0/1047820533 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044011790 con 0x7f505c068260 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f505affd700 1 -- 172.21.15.118:0/1047820533 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044011b30 con 0x7f505c068260 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 -- 172.21.15.118:0/1047820533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 msgr2=0x7f505c14f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 --2- 172.21.15.118:0/1047820533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c14f130 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f50440098e0 tx=0x7f50440091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 -- 172.21.15.118:0/1047820533 shutdown_connections 2024-08-18T13:39:41.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 --2- 172.21.15.118:0/1047820533 >> v2:172.21.15.178:3300/0 conn(0x7f505c0897f0 0x7f505c14f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 --2- 172.21.15.118:0/1047820533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c14f130 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 -- 172.21.15.118:0/1047820533 >> 172.21.15.118:0/1047820533 conn(0x7f505c067e60 msgr2=0x7f505c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:41.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 -- 172.21.15.118:0/1047820533 shutdown_connections 2024-08-18T13:39:41.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 -- 172.21.15.118:0/1047820533 wait complete. 2024-08-18T13:39:41.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 Processor -- start 2024-08-18T13:39:41.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.542+0000 7f506227e700 1 -- start start 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c169220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f505c0897f0 0x7f505c14d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f505c0c0cf0 0x7f505c165a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c1652e0 con 0x7f505c0c0cf0 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c165580 con 0x7f505c0897f0 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c152c40 con 0x7f505c068260 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f505c0c0cf0 0x7f505c165a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f505c0c0cf0 0x7f505c165a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54168/0 (socket says 172.21.15.118:54168) 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 -- 172.21.15.118:0/3997673818 learned_addr learned my addr 172.21.15.118:0/3997673818 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f505bfff700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c169220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f505b7fe700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f505c0897f0 0x7f505c14d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 -- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 msgr2=0x7f505c169220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c169220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 -- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f505c0897f0 msgr2=0x7f505c14d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f505c0897f0 0x7f505c14d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 -- 172.21.15.118:0/3997673818 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044009420 con 0x7f505c0c0cf0 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506081b700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f505c0c0cf0 0x7f505c165a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5054009f20 tx=0x7f505400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f5053fff700 1 -- 172.21.15.118:0/3997673818 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505400cb70 con 0x7f505c0c0cf0 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f5053fff700 1 -- 172.21.15.118:0/3997673818 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505400f040 con 0x7f505c0c0cf0 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c151220 con 0x7f505c0c0cf0 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f5053fff700 1 -- 172.21.15.118:0/3997673818 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054014480 con 0x7f505c0c0cf0 2024-08-18T13:39:41.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c152960 con 0x7f505c0c0cf0 2024-08-18T13:39:41.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.546+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c165730 con 0x7f505c0c0cf0 2024-08-18T13:39:41.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.550+0000 7f5053fff700 1 -- 172.21.15.118:0/3997673818 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5054014830 con 0x7f505c0c0cf0 2024-08-18T13:39:41.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.550+0000 7f5053fff700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5040060860 0x7f5040062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.550+0000 7f5053fff700 1 -- 172.21.15.118:0/3997673818 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f505407db00 con 0x7f505c0c0cf0 2024-08-18T13:39:41.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.550+0000 7f505bfff700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5040060860 0x7f5040062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.550+0000 7f505bfff700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5040060860 0x7f5040062d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f504c007fd0 tx=0x7f504c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.558+0000 7f5053fff700 1 -- 172.21.15.118:0/3997673818 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f505c165730 con 0x7f505c0c0cf0 2024-08-18T13:39:41.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.750+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f505c165730 con 0x7f505c0c0cf0 2024-08-18T13:39:41.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f5053fff700 1 -- 172.21.15.118:0/3997673818 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+6123 (secure 0 0 0) 0x7f505c165730 con 0x7f505c0c0cf0 2024-08-18T13:39:41.758 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:41.758 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:40.441441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5040060860 msgr2=0x7f5040062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5040060860 0x7f5040062d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f504c007fd0 tx=0x7f504c009040 comp rx=0 tx=0).stop 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f505c0c0cf0 msgr2=0x7f505c165a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f505c0c0cf0 0x7f505c165a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5054009f20 tx=0x7f505400b040 comp rx=0 tx=0).stop 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 shutdown_connections 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5040060860 0x7f5040062d10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f505c0c0cf0 0x7f505c165a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f505c0897f0 0x7f505c14d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 --2- 172.21.15.118:0/3997673818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f505c068260 0x7f505c169220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 >> 172.21.15.118:0/3997673818 conn(0x7f505c067e60 msgr2=0x7f505c14ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:41.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 shutdown_connections 2024-08-18T13:39:41.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.754+0000 7f506227e700 1 -- 172.21.15.118:0/3997673818 wait complete. 2024-08-18T13:39:41.762 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 6 2024-08-18T13:39:41.771 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:41.871 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f25b0700 1 Processor -- start 2024-08-18T13:39:41.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f25b0700 1 -- start start 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f25b0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f13ec06ca70 0x7f13ec14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f25b0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f13ec0c4fc0 0x7f13ec14f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f25b0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec068290 con 0x7f13ec06c6a0 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f25b0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec068e30 con 0x7f13ec06ca70 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f25b0700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec067f60 con 0x7f13ec0c4fc0 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f0dad700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f13ec06ca70 0x7f13ec14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f0dad700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f13ec06ca70 0x7f13ec14ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54172/0 (socket says 172.21.15.118:54172) 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f0dad700 1 -- 172.21.15.118:0/389819144 learned_addr learned my addr 172.21.15.118:0/389819144 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:41.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f1daf700 1 --2- 172.21.15.118:0/389819144 >> v2:172.21.15.178:3300/0 conn(0x7f13ec0c4fc0 0x7f13ec14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f0dad700 1 -- 172.21.15.118:0/389819144 >> v2:172.21.15.178:3300/0 conn(0x7f13ec0c4fc0 msgr2=0x7f13ec14f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:41.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f0dad700 1 --2- 172.21.15.118:0/389819144 >> v2:172.21.15.178:3300/0 conn(0x7f13ec0c4fc0 0x7f13ec14f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f0dad700 1 -- 172.21.15.118:0/389819144 >> v1:172.21.15.178:6789/0 conn(0x7f13ec06c6a0 legacy=0x7f13ec068720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:41.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.866+0000 7f13f0dad700 1 -- 172.21.15.118:0/389819144 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec159430 con 0x7f13ec06ca70 2024-08-18T13:39:41.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f1daf700 1 --2- 172.21.15.118:0/389819144 >> v2:172.21.15.178:3300/0 conn(0x7f13ec0c4fc0 0x7f13ec14f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:39:41.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f0dad700 1 --2- 172.21.15.118:0/389819144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06ca70 0x7f13ec14ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13ec06bc10 tx=0x7f13e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca3fff05363d1e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13e3fff700 1 -- 172.21.15.118:0/389819144 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800b820 con 0x7f13ec06ca70 2024-08-18T13:39:41.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13e3fff700 1 -- 172.21.15.118:0/389819144 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e800b980 con 0x7f13ec06ca70 2024-08-18T13:39:41.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13e3fff700 1 -- 172.21.15.118:0/389819144 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800bd20 con 0x7f13ec06ca70 2024-08-18T13:39:41.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 -- 172.21.15.118:0/389819144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06ca70 msgr2=0x7f13ec14ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 --2- 172.21.15.118:0/389819144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06ca70 0x7f13ec14ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13ec06bc10 tx=0x7f13e8009250 comp rx=0 tx=0).stop 2024-08-18T13:39:41.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 -- 172.21.15.118:0/389819144 shutdown_connections 2024-08-18T13:39:41.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 --2- 172.21.15.118:0/389819144 >> v2:172.21.15.178:3300/0 conn(0x7f13ec0c4fc0 0x7f13ec14f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 --2- 172.21.15.118:0/389819144 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06ca70 0x7f13ec14ee30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 -- 172.21.15.118:0/389819144 >> 172.21.15.118:0/389819144 conn(0x7f13ec06f4d0 msgr2=0x7f13ec0c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:41.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 -- 172.21.15.118:0/389819144 shutdown_connections 2024-08-18T13:39:41.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.870+0000 7f13f25b0700 1 -- 172.21.15.118:0/389819144 wait complete. 2024-08-18T13:39:41.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 Processor -- start 2024-08-18T13:39:41.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 -- start start 2024-08-18T13:39:41.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06c6a0 0x7f13ec1688f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13ec06ca70 0x7f13ec0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13ec0c4fc0 0x7f13ec1504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1515a0 con 0x7f13ec06c6a0 2024-08-18T13:39:41.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec152c50 con 0x7f13ec0c4fc0 2024-08-18T13:39:41.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec154620 con 0x7f13ec06ca70 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06c6a0 0x7f13ec1688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06c6a0 0x7f13ec1688f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54176/0 (socket says 172.21.15.118:54176) 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 -- 172.21.15.118:0/2457380000 learned_addr learned my addr 172.21.15.118:0/2457380000 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f0dad700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13ec06ca70 0x7f13ec0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f1daf700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13ec0c4fc0 0x7f13ec1504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 -- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13ec06ca70 msgr2=0x7f13ec0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13ec06ca70 0x7f13ec0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 -- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13ec0c4fc0 msgr2=0x7f13ec1504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13ec0c4fc0 0x7f13ec1504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 -- 172.21.15.118:0/2457380000 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80094f0 con 0x7f13ec06c6a0 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f1daf700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13ec0c4fc0 0x7f13ec1504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:41.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f15ae700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06c6a0 0x7f13ec1688f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00b5e0 tx=0x7f13dc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13e17fa700 1 -- 172.21.15.118:0/2457380000 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00f510 con 0x7f13ec06c6a0 2024-08-18T13:39:41.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13e17fa700 1 -- 172.21.15.118:0/2457380000 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc00f670 con 0x7f13ec06c6a0 2024-08-18T13:39:41.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec154900 con 0x7f13ec06c6a0 2024-08-18T13:39:41.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13e17fa700 1 -- 172.21.15.118:0/2457380000 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00fa30 con 0x7f13ec06c6a0 2024-08-18T13:39:41.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.874+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec165760 con 0x7f13ec06c6a0 2024-08-18T13:39:41.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.878+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec0713a0 con 0x7f13ec06c6a0 2024-08-18T13:39:41.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.878+0000 7f13e17fa700 1 -- 172.21.15.118:0/2457380000 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f13dc01a400 con 0x7f13ec06c6a0 2024-08-18T13:39:41.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.878+0000 7f13e17fa700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f13d0060860 0x7f13d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:41.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.878+0000 7f13e17fa700 1 -- 172.21.15.118:0/2457380000 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f13dc07c9b0 con 0x7f13ec06c6a0 2024-08-18T13:39:41.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.878+0000 7f13f0dad700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f13d0060860 0x7f13d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:41.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.878+0000 7f13f0dad700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f13d0060860 0x7f13d0062d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13ec06bc10 tx=0x7f13e8016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:41.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:41.886+0000 7f13e17fa700 1 -- 172.21.15.118:0/2457380000 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f13dc04cb90 con 0x7f13ec06c6a0 2024-08-18T13:39:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.050+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13ec0bf6d0 con 0x7f13ec06c6a0 2024-08-18T13:39:42.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13e17fa700 1 -- 172.21.15.118:0/2457380000 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f13dc0501b0 con 0x7f13ec06c6a0 2024-08-18T13:39:42.058 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:42.058 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f13d0060860 msgr2=0x7f13d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f13d0060860 0x7f13d0062d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13ec06bc10 tx=0x7f13e8016000 comp rx=0 tx=0).stop 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06c6a0 msgr2=0x7f13ec1688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06c6a0 0x7f13ec1688f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00b5e0 tx=0x7f13dc009120 comp rx=0 tx=0).stop 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 shutdown_connections 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f13d0060860 0x7f13d0062d10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13ec0c4fc0 0x7f13ec1504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13ec06ca70 0x7f13ec0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 --2- 172.21.15.118:0/2457380000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13ec06c6a0 0x7f13ec1688f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 >> 172.21.15.118:0/2457380000 conn(0x7f13ec06f4d0 msgr2=0x7f13ec068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 shutdown_connections 2024-08-18T13:39:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.054+0000 7f13f25b0700 1 -- 172.21.15.118:0/2457380000 wait complete. 2024-08-18T13:39:42.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-18T13:39:34.735047+0000', 'modified': '2024-08-18T13:39:40.441441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4312': {'gid': 4312, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.178:6837/2503344632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2503344632}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2503344632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 1} 2024-08-18T13:39:42.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-18T13:39:42.679 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-08-18T13:39:42.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba57fff700 1 Processor -- start 2024-08-18T13:39:42.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba57fff700 1 -- start start 2024-08-18T13:39:42.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba57fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fba580c0b00 0x7fba58068820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba57fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fba58155c90 0x7fba580c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba57fff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba580c15e0 con 0x7fba580c8d40 2024-08-18T13:39:42.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba57fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba580c7bf0 con 0x7fba580c0b00 2024-08-18T13:39:42.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba57fff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba581561b0 con 0x7fba58155c90 2024-08-18T13:39:42.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba56ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fba580c0b00 0x7fba58068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba56ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fba580c0b00 0x7fba58068820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54190/0 (socket says 172.21.15.118:54190) 2024-08-18T13:39:42.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba56ffd700 1 -- 172.21.15.118:0/3120107934 learned_addr learned my addr 172.21.15.118:0/3120107934 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:42.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba567fc700 1 --2- 172.21.15.118:0/3120107934 >> v2:172.21.15.178:3300/0 conn(0x7fba58155c90 0x7fba580c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba56ffd700 1 -- 172.21.15.118:0/3120107934 >> v2:172.21.15.178:3300/0 conn(0x7fba58155c90 msgr2=0x7fba580c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:42.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba56ffd700 1 --2- 172.21.15.118:0/3120107934 >> v2:172.21.15.178:3300/0 conn(0x7fba58155c90 0x7fba580c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba56ffd700 1 -- 172.21.15.118:0/3120107934 >> v1:172.21.15.178:6789/0 conn(0x7fba580c8d40 legacy=0x7fba580c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.794+0000 7fba56ffd700 1 -- 172.21.15.118:0/3120107934 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580c7000 con 0x7fba580c0b00 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba56ffd700 1 --2- 172.21.15.118:0/3120107934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58068820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba580703c0 tx=0x7fba50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5e9726f0b3633f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba55ffb700 1 -- 172.21.15.118:0/3120107934 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50009e70 con 0x7fba580c0b00 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba55ffb700 1 -- 172.21.15.118:0/3120107934 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5000b8f0 con 0x7fba580c0b00 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba55ffb700 1 -- 172.21.15.118:0/3120107934 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5000bc90 con 0x7fba580c0b00 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 -- 172.21.15.118:0/3120107934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 msgr2=0x7fba58068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 --2- 172.21.15.118:0/3120107934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58068820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba580703c0 tx=0x7fba50009250 comp rx=0 tx=0).stop 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 -- 172.21.15.118:0/3120107934 shutdown_connections 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 --2- 172.21.15.118:0/3120107934 >> v2:172.21.15.178:3300/0 conn(0x7fba58155c90 0x7fba580c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 --2- 172.21.15.118:0/3120107934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58068820 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 -- 172.21.15.118:0/3120107934 >> 172.21.15.118:0/3120107934 conn(0x7fba580cc630 msgr2=0x7fba580c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 -- 172.21.15.118:0/3120107934 shutdown_connections 2024-08-18T13:39:42.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 -- 172.21.15.118:0/3120107934 wait complete. 2024-08-18T13:39:42.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 Processor -- start 2024-08-18T13:39:42.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.798+0000 7fba57fff700 1 -- start start 2024-08-18T13:39:42.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58165ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba580c8d40 0x7fba5815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba58155c90 0x7fba580aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba58166420 con 0x7fba580c0b00 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba580b1370 con 0x7fba58155c90 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba580b04c0 con 0x7fba580c8d40 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58165ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58165ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54200/0 (socket says 172.21.15.118:54200) 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 -- 172.21.15.118:0/1493789965 learned_addr learned my addr 172.21.15.118:0/1493789965 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba567fc700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba580c8d40 0x7fba5815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba577fe700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba58155c90 0x7fba580aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 -- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba580c8d40 msgr2=0x7fba5815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba580c8d40 0x7fba5815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 -- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba58155c90 msgr2=0x7fba580aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:42.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba58155c90 0x7fba580aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 -- 172.21.15.118:0/1493789965 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500094f0 con 0x7fba580c0b00 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba577fe700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba58155c90 0x7fba580aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba567fc700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba580c8d40 0x7fba5815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58165ee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba5804c3c0 tx=0x7fba50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba4effd700 1 -- 172.21.15.118:0/1493789965 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50024440 con 0x7fba580c0b00 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba4effd700 1 -- 172.21.15.118:0/1493789965 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba500245a0 con 0x7fba580c0b00 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba580b0760 con 0x7fba580c0b00 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba4effd700 1 -- 172.21.15.118:0/1493789965 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50024940 con 0x7fba580c0b00 2024-08-18T13:39:42.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.802+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba580b0d20 con 0x7fba580c0b00 2024-08-18T13:39:42.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.806+0000 7fba4effd700 1 -- 172.21.15.118:0/1493789965 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fba5001a420 con 0x7fba580c0b00 2024-08-18T13:39:42.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.806+0000 7fba4effd700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fba38060860 0x7fba38062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.806+0000 7fba4effd700 1 -- 172.21.15.118:0/1493789965 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fba5007ef30 con 0x7fba580c0b00 2024-08-18T13:39:42.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.806+0000 7fba567fc700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fba38060860 0x7fba38062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.806+0000 7fba567fc700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fba38060860 0x7fba38062d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba40006f00 tx=0x7fba40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:42.813 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 Processor -- start 2024-08-18T13:39:42.813 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- start start 2024-08-18T13:39:42.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x285d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x285c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x285c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x27d4ba0 con 0x2628ae0 2024-08-18T13:39:42.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x27d0b40 con 0x2628710 2024-08-18T13:39:42.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x27960a0 con 0x2673790 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x285d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x285d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54214/0 (socket says 172.21.15.118:54214) 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 -- 172.21.15.118:0/1357792624 learned_addr learned my addr 172.21.15.118:0/1357792624 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba567fc700 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x285c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba577fe700 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x285c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 -- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 msgr2=0x285c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x285c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 -- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 msgr2=0x285c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x285c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 -- 172.21.15.118:0/1357792624 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26749e0 con 0x2628ae0 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x285d2a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba5007d130 tx=0x7fba5007d0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e899b3c5e37009f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba37fff700 1 -- 172.21.15.118:0/1357792624 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500580f0 con 0x2628ae0 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba37fff700 1 -- 172.21.15.118:0/1357792624 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50053490 con 0x2628ae0 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba37fff700 1 -- 172.21.15.118:0/1357792624 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50055fe0 con 0x2628ae0 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- 172.21.15.118:0/1357792624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 msgr2=0x285d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x285d2a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba5007d130 tx=0x7fba5007d0c0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- 172.21.15.118:0/1357792624 shutdown_connections 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x285c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x285c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 --2- 172.21.15.118:0/1357792624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x285d2a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- 172.21.15.118:0/1357792624 >> 172.21.15.118:0/1357792624 conn(0x2628eb0 msgr2=0x289b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:42.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- 172.21.15.118:0/1357792624 shutdown_connections 2024-08-18T13:39:42.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.810+0000 7fba5f625740 1 -- 172.21.15.118:0/1357792624 wait complete. 2024-08-18T13:39:42.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 -- start start 2024-08-18T13:39:42.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 Processor -- start 2024-08-18T13:39:42.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628ae0 0x28a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2673790 0x2860030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x2795d40 con 0x2673790 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x276e970 con 0x2628710 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x27d4ba0 con 0x2628ae0 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2673790 0x2860030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2673790 0x2860030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54226/0 (socket says 172.21.15.118:54226) 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 -- 172.21.15.118:0/1808873486 learned_addr learned my addr 172.21.15.118:0/1808873486 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba56ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba567fc700 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628ae0 0x28a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 10 client.0 _ll_register_callbacks cb 0x2772f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 -- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628ae0 msgr2=0x28a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628ae0 0x28a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 -- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 msgr2=0x28a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:42.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 -- 172.21.15.118:0/1808873486 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba580b0d20 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba577fe700 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2673790 0x2860030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba4800f880 tx=0x7fba48009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaf544220cf3489 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba357fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba357fa700 1 -- 172.21.15.118:0/1808873486 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48001bb0 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba357fa700 1 -- 172.21.15.118:0/1808873486 <== mon.0 v2:172.21.15.118:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fba48001d10 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.814+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2636260 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba357fa700 1 -- 172.21.15.118:0/1808873486 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba48002260 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 10 client.4413 fetch_fsmap learned FSMap version 6 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x279e370 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba357fa700 1 -- 172.21.15.118:0/1808873486 <== mon.0 v2:172.21.15.118:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2636260 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba357fa700 1 -- 172.21.15.118:0/1808873486 <== mon.0 v2:172.21.15.118:3300/0 5 ==== fsmap(e 6) v1 ==== 4198+0+0 (secure 0 0 0) 0x7fba48012070 con 0x2673790 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 10 client.4413 fetch_fsmap finished waiting for FSMap version 6 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 10 client.4413 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 20 client.4413 populate_metadata read hostname 'smithi118' 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2861650 0x28b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:42.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 4 client.4413 mds_command: new command op to 4312 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-18T13:39:42.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 --> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x26804e0 con 0x2861650 2024-08-18T13:39:42.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2861650 0x28b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:42.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba357fa700 10 client.4413 ms_handle_connect on v2:172.21.15.178:6836/2503344632 2024-08-18T13:39:42.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:42.818+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2861650 0x28b0ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e2d610111f6d1f server_cookie=f3560748dc9c41d7 in_seq=0 out_seq=0 2024-08-18T13:39:45.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba357fa700 1 -- 172.21.15.118:0/1808873486 <== mds.0 v2:172.21.15.178:6836/2503344632 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x26804e0 con 0x2861650 2024-08-18T13:39:45.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba357fa700 10 client.4413 handle_command_reply: tid=0 2024-08-18T13:39:45.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 client.4413 shutdown 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2673790 msgr2=0x2860030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2673790 0x2860030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba4800f880 tx=0x7fba48009040 comp rx=0 tx=0).stop 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 shutdown_connections 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2861650 0x28b0ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2673790 0x2860030 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628ae0 0x28a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba357fa700 0 client.4413 ms_handle_reset on v2:172.21.15.178:6836/2503344632 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 --2- 172.21.15.118:0/1808873486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 >> 172.21.15.118:0/1808873486 conn(0x2628eb0 msgr2=0x2899a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 shutdown_connections 2024-08-18T13:39:45.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 1 -- 172.21.15.118:0/1808873486 wait complete. 2024-08-18T13:39:45.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.450+0000 7fba5f625740 20 client.4413 trim_cache size 0 max 16384 2024-08-18T13:39:45.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 Processor -- start 2024-08-18T13:39:45.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 -- start start 2024-08-18T13:39:45.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x28f3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x27d6da0 con 0x2628ae0 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x27cc650 con 0x2628710 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x27cf9d0 con 0x2673790 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba567fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba567fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47528/0 (socket says 172.21.15.118:47528) 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba567fc700 1 -- 172.21.15.118:0/1332746428 learned_addr learned my addr 172.21.15.118:0/1332746428 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba577fe700 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x28f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.474+0000 7fba56ffd700 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba567fc700 1 -- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 msgr2=0x28f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba567fc700 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x28f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba567fc700 1 -- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 msgr2=0x28a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba567fc700 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba567fc700 1 -- 172.21.15.118:0/1332746428 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2784d80 con 0x2628ae0 2024-08-18T13:39:45.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba567fc700 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28a4780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba40010ce0 tx=0x7fba40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb716a55d8491a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:45.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba4cff9700 1 -- 172.21.15.118:0/1332746428 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40015c10 con 0x2628ae0 2024-08-18T13:39:45.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba4cff9700 1 -- 172.21.15.118:0/1332746428 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba40015d70 con 0x2628ae0 2024-08-18T13:39:45.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba4cff9700 1 -- 172.21.15.118:0/1332746428 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40024650 con 0x2628ae0 2024-08-18T13:39:45.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 -- 172.21.15.118:0/1332746428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 msgr2=0x28a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28a4780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba40010ce0 tx=0x7fba40017000 comp rx=0 tx=0).stop 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 -- 172.21.15.118:0/1332746428 shutdown_connections 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2673790 0x28f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28a4780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 --2- 172.21.15.118:0/1332746428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2628710 0x28a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 -- 172.21.15.118:0/1332746428 >> 172.21.15.118:0/1332746428 conn(0x2628eb0 msgr2=0x28a3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 -- 172.21.15.118:0/1332746428 shutdown_connections 2024-08-18T13:39:45.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.478+0000 7fba5f625740 1 -- 172.21.15.118:0/1332746428 wait complete. 2024-08-18T13:39:45.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 -- start start 2024-08-18T13:39:45.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 Processor -- start 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628710 0x28fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2673790 0x28bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x2799090 con 0x2628ae0 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x2772d30 con 0x2673790 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x27d6f00 con 0x2628710 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba5f625740 10 client.0 _ll_register_callbacks cb 0x26f7d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba567fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba56ffd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628710 0x28fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba567fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28fd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47540/0 (socket says 172.21.15.118:47540) 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba567fc700 1 -- 172.21.15.118:0/901713502 learned_addr learned my addr 172.21.15.118:0/901713502 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba577fe700 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2673790 0x28bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba56ffd700 1 -- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2673790 msgr2=0x28bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba56ffd700 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2673790 0x28bcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba56ffd700 1 -- 172.21.15.118:0/901713502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 msgr2=0x28fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba56ffd700 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba56ffd700 1 -- 172.21.15.118:0/901713502 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4000e050 con 0x2628710 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba567fc700 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba56ffd700 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628710 0x28fcf90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fba50052ce0 tx=0x7fba50008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d21314447cf5f5e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:45.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.482+0000 7fba357fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3301/0 2024-08-18T13:39:45.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 1 -- 172.21.15.118:0/901713502 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500510f0 con 0x2628710 2024-08-18T13:39:45.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba50053490 con 0x2628710 2024-08-18T13:39:45.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 1 -- 172.21.15.118:0/901713502 <== mon.2 v2:172.21.15.178:3301/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fba50068170 con 0x2628710 2024-08-18T13:39:45.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 1 -- 172.21.15.118:0/901713502 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba50068330 con 0x2628710 2024-08-18T13:39:45.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 10 client.4358 fetch_fsmap learned FSMap version 6 2024-08-18T13:39:45.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x2779130 con 0x2628710 2024-08-18T13:39:45.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 1 -- 172.21.15.118:0/901713502 <== mon.2 v2:172.21.15.178:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50053490 con 0x2628710 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 1 -- 172.21.15.118:0/901713502 <== mon.2 v2:172.21.15.178:3301/0 5 ==== fsmap(e 6) v1 ==== 4198+0+0 (secure 0 0 0) 0x7fba50056d20 con 0x2628710 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 10 client.4358 fetch_fsmap finished waiting for FSMap version 6 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 10 client.4358 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 20 client.4358 populate_metadata read hostname 'smithi118' 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2903e20 0x28bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 4 client.4358 mds_command: new command op to 4312 tid=0[{"prefix": "status"}] 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 --> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] -- command(tid 0: {"prefix": "status"}) v1 -- 0x26a5260 con 0x2903e20 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba567fc700 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2903e20 0x28bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 10 client.4358 ms_handle_connect on v2:172.21.15.178:6836/2503344632 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba567fc700 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2903e20 0x28bd880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430d7e8349a4e63e server_cookie=52938b07decca39d in_seq=0 out_seq=0 2024-08-18T13:39:45.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 1 -- 172.21.15.118:0/901713502 <== mds.0 v2:172.21.15.178:6836/2503344632 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x26a5260 con 0x2903e20 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 10 client.4358 handle_command_reply: tid=0 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 client.4358 shutdown 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628710 msgr2=0x28fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628710 0x28fcf90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fba50052ce0 tx=0x7fba50008040 comp rx=0 tx=0).stop 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 shutdown_connections 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x2903e20 0x28bd880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba357fa700 0 client.4358 ms_handle_reset on v2:172.21.15.178:6836/2503344632 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x2673790 0x28bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x2628ae0 0x28fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 --2- 172.21.15.118:0/901713502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x2628710 0x28fcf90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 >> 172.21.15.118:0/901713502 conn(0x2628eb0 msgr2=0x28abdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:45.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 shutdown_connections 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 1 -- 172.21.15.118:0/901713502 wait complete. 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.486+0000 7fba5f625740 20 client.4358 trim_cache size 0 max 16384 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_fsid": "ac1e0c0e-66a6-4fa9-9612-9b35e09366c7", 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stdout: "whoami": 0, 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4312, 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stdout: "want_state": "up:active", 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stdout: "state": "up:active", 2024-08-18T13:39:45.496 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs", 2024-08-18T13:39:45.497 INFO:teuthology.orchestra.run.smithi118.stdout: "rank_uptime": 10.076858867, 2024-08-18T13:39:45.497 INFO:teuthology.orchestra.run.smithi118.stdout: "mdsmap_epoch": 5, 2024-08-18T13:39:45.497 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_epoch": 25, 2024-08-18T13:39:45.497 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_epoch_barrier": 25, 2024-08-18T13:39:45.497 INFO:teuthology.orchestra.run.smithi118.stdout: "uptime": 17.147897145999998 2024-08-18T13:39:45.497 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fba38060860 msgr2=0x7fba38062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fba38060860 0x7fba38062d10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba40006f00 tx=0x7fba40008040 comp rx=0 tx=0).stop 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 msgr2=0x7fba58165ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58165ee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba5804c3c0 tx=0x7fba50017000 comp rx=0 tx=0).stop 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 shutdown_connections 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fba38060860 0x7fba38062d10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba58155c90 0x7fba580aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba580c8d40 0x7fba5815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 --2- 172.21.15.118:0/1493789965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba580c0b00 0x7fba58165ee0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 >> 172.21.15.118:0/1493789965 conn(0x7fba580cc630 msgr2=0x7fba5814ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:45.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 shutdown_connections 2024-08-18T13:39:45.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.490+0000 7fba57fff700 1 -- 172.21.15.118:0/1493789965 wait complete. 2024-08-18T13:39:45.510 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-18T13:39:45.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e1bf9700 1 Processor -- start 2024-08-18T13:39:45.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e1bf9700 1 -- start start 2024-08-18T13:39:45.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e1bf9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f82dc0aebc0 0x7f82dc0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e1bf9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82dc0c6880 0x7f82dc0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e1bf9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc155330 con 0x7f82dc0aef90 2024-08-18T13:39:45.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e1bf9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc155140 con 0x7f82dc0aebc0 2024-08-18T13:39:45.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e1bf9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc0ac1e0 con 0x7f82dc0c6880 2024-08-18T13:39:45.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f82dc0aebc0 0x7f82dc0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f82dc0aebc0 0x7f82dc0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47556/0 (socket says 172.21.15.118:47556) 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 -- 172.21.15.118:0/4038320281 learned_addr learned my addr 172.21.15.118:0/4038320281 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e13f8700 1 --2- 172.21.15.118:0/4038320281 >> v2:172.21.15.178:3300/0 conn(0x7f82dc0c6880 0x7f82dc0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 -- 172.21.15.118:0/4038320281 >> v2:172.21.15.178:3300/0 conn(0x7f82dc0c6880 msgr2=0x7f82dc0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 --2- 172.21.15.118:0/4038320281 >> v2:172.21.15.178:3300/0 conn(0x7f82dc0c6880 0x7f82dc0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 -- 172.21.15.118:0/4038320281 >> v1:172.21.15.178:6789/0 conn(0x7f82dc0aef90 legacy=0x7f82dc0abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 -- 172.21.15.118:0/4038320281 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc1555d0 con 0x7f82dc0aebc0 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82e0bf7700 1 --2- 172.21.15.118:0/4038320281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0aebc0 0x7f82dc0b1640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0ccbf0 tx=0x7f82c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28d06d34745fe92 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82db7fe700 1 -- 172.21.15.118:0/4038320281 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400b820 con 0x7f82dc0aebc0 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82db7fe700 1 -- 172.21.15.118:0/4038320281 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c400b980 con 0x7f82dc0aebc0 2024-08-18T13:39:45.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.622+0000 7f82db7fe700 1 -- 172.21.15.118:0/4038320281 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400bd20 con 0x7f82dc0aebc0 2024-08-18T13:39:45.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 -- 172.21.15.118:0/4038320281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0aebc0 msgr2=0x7f82dc0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/4038320281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0aebc0 0x7f82dc0b1640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0ccbf0 tx=0x7f82c4009250 comp rx=0 tx=0).stop 2024-08-18T13:39:45.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 -- 172.21.15.118:0/4038320281 shutdown_connections 2024-08-18T13:39:45.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/4038320281 >> v2:172.21.15.178:3300/0 conn(0x7f82dc0c6880 0x7f82dc0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/4038320281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0aebc0 0x7f82dc0b1640 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 -- 172.21.15.118:0/4038320281 >> 172.21.15.118:0/4038320281 conn(0x7f82dc0cbba0 msgr2=0x7f82dc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:45.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 -- 172.21.15.118:0/4038320281 shutdown_connections 2024-08-18T13:39:45.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 -- 172.21.15.118:0/4038320281 wait complete. 2024-08-18T13:39:45.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 Processor -- start 2024-08-18T13:39:45.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.626+0000 7f82e1bf9700 1 -- start start 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82dc0aebc0 0x7f82dc168f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82dc0aef90 0x7f82dc164fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0c6880 0x7f82dc0c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc168ae0 con 0x7f82dc0c6880 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc1684d0 con 0x7f82dc0aef90 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c23d0 con 0x7f82dc0aebc0 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0c6880 0x7f82dc0c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0c6880 0x7f82dc0c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47572/0 (socket says 172.21.15.118:47572) 2024-08-18T13:39:45.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 -- 172.21.15.118:0/1637466740 learned_addr learned my addr 172.21.15.118:0/1637466740 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e0bf7700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82dc0aebc0 0x7f82dc168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82dbfff700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82dc0aef90 0x7f82dc164fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 -- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82dc0aebc0 msgr2=0x7f82dc168f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82dc0aebc0 0x7f82dc168f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 -- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82dc0aef90 msgr2=0x7f82dc164fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82dc0aef90 0x7f82dc164fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 -- 172.21.15.118:0/1637466740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40094f0 con 0x7f82dc0c6880 2024-08-18T13:39:45.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e13f8700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0c6880 0x7f82dc0c8a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00d860 tx=0x7f82cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:45.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82d8ff9700 1 -- 172.21.15.118:0/1637466740 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0125f0 con 0x7f82dc0c6880 2024-08-18T13:39:45.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82d8ff9700 1 -- 172.21.15.118:0/1637466740 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc012750 con 0x7f82dc0c6880 2024-08-18T13:39:45.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc0c0b30 con 0x7f82dc0c6880 2024-08-18T13:39:45.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82d8ff9700 1 -- 172.21.15.118:0/1637466740 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc012b10 con 0x7f82dc0c6880 2024-08-18T13:39:45.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.630+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc168770 con 0x7f82dc0c6880 2024-08-18T13:39:45.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.634+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc168c90 con 0x7f82dc0c6880 2024-08-18T13:39:45.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.634+0000 7f82d8ff9700 1 -- 172.21.15.118:0/1637466740 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f82cc013480 con 0x7f82dc0c6880 2024-08-18T13:39:45.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.634+0000 7f82d8ff9700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82c0060890 0x7f82c0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:45.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.634+0000 7f82d8ff9700 1 -- 172.21.15.118:0/1637466740 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f82cc07d130 con 0x7f82dc0c6880 2024-08-18T13:39:45.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.634+0000 7f82e0bf7700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82c0060890 0x7f82c0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:45.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.634+0000 7f82e0bf7700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82c0060890 0x7f82c0062d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0ccbf0 tx=0x7f82c4016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:45.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.638+0000 7f82d8ff9700 1 -- 172.21.15.118:0/1637466740 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f82dc168c90 con 0x7f82dc0c6880 2024-08-18T13:39:45.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.834+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f82dc0c0fb0 con 0x7f82dc0c6880 2024-08-18T13:39:45.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82d8ff9700 1 -- 172.21.15.118:0/1637466740 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f82cc04d1f0 con 0x7f82dc0c6880 2024-08-18T13:39:45.842 INFO:teuthology.orchestra.run.smithi118.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-18T13:39:45.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82c0060890 msgr2=0x7f82c0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82c0060890 0x7f82c0062d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0ccbf0 tx=0x7f82c4016000 comp rx=0 tx=0).stop 2024-08-18T13:39:45.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0c6880 msgr2=0x7f82dc0c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:45.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0c6880 0x7f82dc0c8a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00d860 tx=0x7f82cc008040 comp rx=0 tx=0).stop 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 shutdown_connections 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82c0060890 0x7f82c0062d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82dc0c6880 0x7f82dc0c8a80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82dc0aef90 0x7f82dc164fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 --2- 172.21.15.118:0/1637466740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82dc0aebc0 0x7f82dc168f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 >> 172.21.15.118:0/1637466740 conn(0x7f82dc0cbba0 msgr2=0x7f82dc1544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 shutdown_connections 2024-08-18T13:39:45.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:45.838+0000 7f82e1bf9700 1 -- 172.21.15.118:0/1637466740 wait complete. 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi118.front.sepia.ceph.com 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-18T13:39:45.856 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-08-18T13:39:45.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:45.856 DEBUG:teuthology.orchestra.run.smithi118:> ip addr 2024-08-18T13:39:45.864 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-18T13:39:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-18T13:39:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2024-08-18T13:39:45.864 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cc brd ff:ff:ff:ff:ff:ff 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cd brd ff:ff:ff:ff:ff:ff 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:24 brd ff:ff:ff:ff:ff:ff 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:25 brd ff:ff:ff:ff:ff:ff 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.21.15.118/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft 42603sec preferred_lft 42603sec 2024-08-18T13:39:45.865 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2024-08-18T13:39:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:39:45.866 INFO:teuthology.orchestra.run.smithi118.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-18T13:39:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 02:42:27:94:1d:74 brd ff:ff:ff:ff:ff:ff 2024-08-18T13:39:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-18T13:39:45.866 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:39:45.866 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-18T13:39:45.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:45.867 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:39:45.867 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:39:45.867 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link add name ceph-brx type bridge 2024-08-18T13:39:45.867 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip addr flush dev ceph-brx 2024-08-18T13:39:45.867 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set ceph-brx up 2024-08-18T13:39:45.867 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-18T13:39:45.867 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:39:45.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:45.911 DEBUG:teuthology.orchestra.run.smithi118:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-18T13:39:45.971 INFO:teuthology.orchestra.run.smithi118.stdout:1 2024-08-18T13:39:45.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:45.973 DEBUG:teuthology.orchestra.run.smithi118:> route 2024-08-18T13:39:46.002 INFO:teuthology.orchestra.run.smithi118.stdout:Kernel IP routing table 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run.smithi118.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run.smithi118.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run.smithi118.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run.smithi118.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-18T13:39:46.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:46.004 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:39:46.004 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:39:46.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-18T13:39:46.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-18T13:39:46.004 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-18T13:39:46.004 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:39:46.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:46.046 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2024-08-18T13:39:46.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:46.096 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list-id 2024-08-18T13:39:46.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:46.104 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:39:46.104 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:39:46.104 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T13:39:46.104 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-08-18T13:39:46.104 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:39:46.135 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-08-18T13:39:46.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-08-18T13:39:46.136 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:39:46.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:46.208 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:39:46.208 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:39:46.208 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set brx.0 up 2024-08-18T13:39:46.208 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set dev brx.0 master ceph-brx 2024-08-18T13:39:46.209 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:39:46.236 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-08-18T13:39:46.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T13:39:46.236 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-08-18T13:39:46.242 INFO:teuthology.orchestra.run.smithi118.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-08-18T13:39:46.242 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T13:39:46.242 DEBUG:teuthology.orchestra.run.smithi118:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-08-18T13:39:46.247 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2024-08-18T13:39:46.262 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-18T13:39:46.262 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-18T13:39:46.275 INFO:teuthology.orchestra.run.smithi118.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-18T13:39:46.275 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-18T13:39:46.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:39:46.276 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2024-08-18T13:39:46.282 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-18T13:39:46.282 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-08-18T13:39:46.327 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2024-08-18T13:39:46.345 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-18T13:39:46.345 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-18T13:39:46.354 INFO:teuthology.orchestra.run.smithi118.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-18T13:39:46.354 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-18T13:39:46.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:39:46.355 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2024-08-18T13:39:46.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi118.stderr:2024-08-18T13:39:46.414+0000 7f3103c4a380 -1 init, newargv = 0x55ae47c6e810 newargc=15 2024-08-18T13:39:46.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi118.stderr:ceph-fuse[30466]: starting ceph client 2024-08-18T13:39:46.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi118.stderr:ceph-fuse[30466]: starting fuse 2024-08-18T13:39:47.504 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2024-08-18T13:39:47.523 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-18T13:39:47.523 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-18T13:39:47.538 INFO:teuthology.orchestra.run.smithi118.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-18T13:39:47.539 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-18T13:39:47.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:39:47.540 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2024-08-18T13:39:47.547 INFO:teuthology.orchestra.run.smithi118.stdout:52 2024-08-18T13:39:47.547 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-18T13:39:47.547 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-18T13:39:47.547 DEBUG:teuthology.orchestra.run.smithi118:> import glob 2024-08-18T13:39:47.547 DEBUG:teuthology.orchestra.run.smithi118:> import re 2024-08-18T13:39:47.547 DEBUG:teuthology.orchestra.run.smithi118:> import os 2024-08-18T13:39:47.547 DEBUG:teuthology.orchestra.run.smithi118:> import subprocess 2024-08-18T13:39:47.547 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:39:47.547 DEBUG:teuthology.orchestra.run.smithi118:> def _find_admin_socket(client_name): 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> files = glob.glob(asok_path) 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> # Given a non-glob path, it better be there 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> if "*" not in asok_path: 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> assert(len(files) == 1) 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> return files[0] 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> for f in files: 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> contents = proc_f.read() 2024-08-18T13:39:47.548 DEBUG:teuthology.orchestra.run.smithi118:> if mountpoint in contents: 2024-08-18T13:39:47.549 DEBUG:teuthology.orchestra.run.smithi118:> return f 2024-08-18T13:39:47.549 DEBUG:teuthology.orchestra.run.smithi118:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-18T13:39:47.549 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:39:47.549 DEBUG:teuthology.orchestra.run.smithi118:> print(_find_admin_socket("client.admin")) 2024-08-18T13:39:47.549 DEBUG:teuthology.orchestra.run.smithi118:> ' 2024-08-18T13:39:47.675 INFO:teuthology.orchestra.run.smithi118.stdout:/var/run/ceph/ceph-client.admin.30466.asok 2024-08-18T13:39:47.860 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.30466.asok 2024-08-18T13:39:47.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:39:47.860 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.30466.asok status 2024-08-18T13:39:47.951 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "metadata": { 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_sha1": "a17c67c052c1476b20f99d847823755b26703128", 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 17.2.7-1375-ga17c67c0 (a17c67c052c1476b20f99d847823755b26703128) quincy (stable)", 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "entity_id": "admin", 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "hostname": "smithi118", 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "pid": "30466", 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "root": "/" 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_count": 0, 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_pinned_count": 0, 2024-08-18T13:39:47.952 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4431, 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "inst": { 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "name": { 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "client", 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "num": 4431 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1335337592 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-08-18T13:39:47.953 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 1335337592 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "inst_str": "client.4431 192.168.0.1:0/1335337592", 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "addr_str": "192.168.0.1:0/1335337592", 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "inode_count": 1, 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_epoch": 5, 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch": 25, 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch_barrier": 0, 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklisted": false, 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs" 2024-08-18T13:39:47.954 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:39:47.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:39:47.961 DEBUG:teuthology.orchestra.run.smithi118:> stat --file-system '--printf=%T 2024-08-18T13:39:47.961 DEBUG:teuthology.orchestra.run.smithi118:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-08-18T13:39:47.968 INFO:teuthology.orchestra.run.smithi118.stdout:fuseblk 2024-08-18T13:39:47.969 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-08-18T13:39:47.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:39:47.969 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-08-18T13:39:48.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:39:48.030 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-08-18T13:39:48.077 INFO:teuthology.orchestra.run.smithi118.stdout:# file: . 2024-08-18T13:39:48.077 INFO:teuthology.orchestra.run.smithi118.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-08-18T13:39:48.077 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:48.078 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com... 2024-08-18T13:39:48.078 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-18T13:39:48.079 DEBUG:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-08-18T13:39:48.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi118.stderr:ceph-fuse[30466]: fuse finished with error 0 and tester_r 0 2024-08-18T13:39:48.098 INFO:teuthology.orchestra.run:waiting for 900 2024-08-18T13:39:50.358 INFO:tasks.ceph.mds.d.smithi178.stderr:2024-08-18T13:39:50.355+0000 7f142ba95700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T13:39:54.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:39:54.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:39:54.199 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-08-18T13:39:54.216 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:54.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.366+0000 7f9b4a738700 1 Processor -- start 2024-08-18T13:39:54.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.366+0000 7f9b4a738700 1 -- start start 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b4a738700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 0x7f9b44156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b4a738700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b441534f0 con 0x7f9b440c4320 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b4a738700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b44156c20 con 0x7f9b440c46f0 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b49736700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f9b440c4320 0x7f9b4405e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45828/0 (socket says 172.21.15.118:45828) 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b4a738700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441574c0 con 0x7f9b44067d50 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b49736700 1 -- 172.21.15.118:0/3599732471 learned_addr learned my addr 172.21.15.118:0/3599732471 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b49f37700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 0x7f9b44156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84965070 0 0) 0x7f9b441534f0 con 0x7f9b440c4320 2024-08-18T13:39:54.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b30003570 con 0x7f9b440c4320 2024-08-18T13:39:54.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3621737248 0 0) 0x7f9b30003570 con 0x7f9b440c4320 2024-08-18T13:39:54.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 msgr2=0x7f9b44156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:54.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 0x7f9b44156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 >> v1:172.21.15.178:6789/0 conn(0x7f9b440c46f0 legacy=0x7f9b4414c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b44156f00 con 0x7f9b440c4320 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f9b40001490 con 0x7f9b440c4320 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 >> v1:172.21.15.118:6789/0 conn(0x7f9b440c4320 legacy=0x7f9b4405e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 0x7f9b30003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b30003fd0 0x7f9b30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b30004d50 0x7f9b300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441574c0 con 0x7f9b30004d50 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3000bb40 con 0x7f9b30003fd0 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3000bd00 con 0x7f9b44067d50 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.370+0000 7f9b49f37700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b30004d50 0x7f9b300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49736700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 0x7f9b30003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b48f35700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b30003fd0 0x7f9b30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49f37700 1 -- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 msgr2=0x7f9b30003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49f37700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b44067d50 0x7f9b30003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49f37700 1 -- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b30003fd0 msgr2=0x7f9b30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49f37700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b30003fd0 0x7f9b30004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49f37700 1 -- 172.21.15.118:0/3599732471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b30003570 con 0x7f9b30004d50 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49f37700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b30004d50 0x7f9b300051c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e210 tx=0x7f9b3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3584e76a7ad71e7b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00eac0 con 0x7f9b30004d50 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00f040 con 0x7f9b30004d50 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b3bfff700 1 -- 172.21.15.118:0/3599732471 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00ee70 con 0x7f9b30004d50 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b4a738700 1 -- 172.21.15.118:0/3599732471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b30004d50 msgr2=0x7f9b300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b4a738700 1 --2- 172.21.15.118:0/3599732471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b30004d50 0x7f9b300051c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e210 tx=0x7f9b3c00b040 comp rx=0 tx=0).stop 2024-08-18T13:39:54.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b49f37700 1 -- 172.21.15.118:0/3599732471 reap_dead start 2024-08-18T13:39:54.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b4a738700 1 -- 172.21.15.118:0/3599732471 shutdown_connections 2024-08-18T13:39:54.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b4a738700 1 -- 172.21.15.118:0/3599732471 >> 172.21.15.118:0/3599732471 conn(0x7f9b440c81e0 msgr2=0x7f9b440c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:54.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b4a738700 1 -- 172.21.15.118:0/3599732471 shutdown_connections 2024-08-18T13:39:54.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.374+0000 7f9b4a738700 1 -- 172.21.15.118:0/3599732471 wait complete. 2024-08-18T13:39:54.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 Processor -- start 2024-08-18T13:39:54.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 -- start start 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b44162570 0x7f9b44152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b4414e820 0x7f9b44162940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44162e80 0x7f9b44149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440b3f20 con 0x7f9b44162570 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44152160 con 0x7f9b44162e80 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44152b80 con 0x7f9b4414e820 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b44162570 0x7f9b44152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b44162570 0x7f9b44152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44872/0 (socket says 172.21.15.118:44872) 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 -- 172.21.15.118:0/119245538 learned_addr learned my addr 172.21.15.118:0/119245538 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49f37700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44162e80 0x7f9b44149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b48f35700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b4414e820 0x7f9b44162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 -- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b4414e820 msgr2=0x7f9b44162940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b4414e820 0x7f9b44162940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 -- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44162e80 msgr2=0x7f9b44149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44162e80 0x7f9b44149ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 -- 172.21.15.118:0/119245538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b44150720 con 0x7f9b44162570 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b49736700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b44162570 0x7f9b44152640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b40004890 tx=0x7f9b40008c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b397fa700 1 -- 172.21.15.118:0/119245538 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4001b390 con 0x7f9b44162570 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b397fa700 1 -- 172.21.15.118:0/119245538 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4001b510 con 0x7f9b44162570 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b44150a00 con 0x7f9b44162570 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b397fa700 1 -- 172.21.15.118:0/119245538 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4001b8d0 con 0x7f9b44162570 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b440b4450 con 0x7f9b44162570 2024-08-18T13:39:54.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.378+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b440b4610 con 0x7f9b44162570 2024-08-18T13:39:54.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.382+0000 7f9b397fa700 1 -- 172.21.15.118:0/119245538 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9b4001bc80 con 0x7f9b44162570 2024-08-18T13:39:54.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.382+0000 7f9b397fa700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b28060860 0x7f9b28062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.382+0000 7f9b397fa700 1 -- 172.21.15.118:0/119245538 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f9b40080ed0 con 0x7f9b44162570 2024-08-18T13:39:54.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.382+0000 7f9b48f35700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b28060860 0x7f9b28062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.382+0000 7f9b48f35700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b28060860 0x7f9b28062d10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b34007e40 tx=0x7f9b34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:54.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.390+0000 7f9b397fa700 1 -- 172.21.15.118:0/119245538 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9b400510e0 con 0x7f9b44162570 2024-08-18T13:39:54.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.554+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b441522a0 con 0x7f9b44162570 2024-08-18T13:39:54.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.558+0000 7f9b397fa700 1 -- 172.21.15.118:0/119245538 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f9b40021090 con 0x7f9b44162570 2024-08-18T13:39:54.562 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:54.562 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b28060860 msgr2=0x7f9b28062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b28060860 0x7f9b28062d10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b34007e40 tx=0x7f9b34009040 comp rx=0 tx=0).stop 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b44162570 msgr2=0x7f9b44152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b44162570 0x7f9b44152640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b40004890 tx=0x7f9b40008c30 comp rx=0 tx=0).stop 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 shutdown_connections 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b28060860 0x7f9b28062d10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44162e80 0x7f9b44149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b4414e820 0x7f9b44162940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 --2- 172.21.15.118:0/119245538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b44162570 0x7f9b44152640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 >> 172.21.15.118:0/119245538 conn(0x7f9b440c81e0 msgr2=0x7f9b440c4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:54.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 shutdown_connections 2024-08-18T13:39:54.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.562+0000 7f9b4a738700 1 -- 172.21.15.118:0/119245538 wait complete. 2024-08-18T13:39:54.581 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:54.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149d25700 1 Processor -- start 2024-08-18T13:39:54.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149d25700 1 -- start start 2024-08-18T13:39:54.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149d25700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb1441561d0 0x7fb14415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149d25700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1440c1110 con 0x7fb1440684e0 2024-08-18T13:39:54.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149d25700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1440c1cb0 con 0x7fb1441556f0 2024-08-18T13:39:54.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149d25700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1440c0b90 con 0x7fb1441561d0 2024-08-18T13:39:54.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149524700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb1441561d0 0x7fb14415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb143fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fb1440684e0 0x7fb144155ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45842/0 (socket says 172.21.15.118:45842) 2024-08-18T13:39:54.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb143fff700 1 -- 172.21.15.118:0/1344296632 learned_addr learned my addr 172.21.15.118:0/1344296632 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:54.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149524700 1 -- 172.21.15.118:0/1344296632 >> v1:172.21.15.178:6789/0 conn(0x7fb1441556f0 legacy=0x7fb1440c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149524700 1 -- 172.21.15.118:0/1344296632 >> v1:172.21.15.118:6789/0 conn(0x7fb1440684e0 legacy=0x7fb144155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149524700 1 -- 172.21.15.118:0/1344296632 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1440c8060 con 0x7fb1441561d0 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb1437fe700 1 -- 172.21.15.118:0/1344296632 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294054188 0 0) 0x7fb1440c1110 con 0x7fb1440684e0 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb149524700 1 --2- 172.21.15.118:0/1344296632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441561d0 0x7fb14415cb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb1340098e0 tx=0x7fb1340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8cc09b5141087a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb1437fe700 1 -- 172.21.15.118:0/1344296632 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134009e80 con 0x7fb1441561d0 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb1437fe700 1 -- 172.21.15.118:0/1344296632 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13400b860 con 0x7fb1441561d0 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.682+0000 7fb1437fe700 1 -- 172.21.15.118:0/1344296632 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400bbe0 con 0x7fb1441561d0 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- 172.21.15.118:0/1344296632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441561d0 msgr2=0x7fb14415cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 --2- 172.21.15.118:0/1344296632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441561d0 0x7fb14415cb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb1340098e0 tx=0x7fb1340091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- 172.21.15.118:0/1344296632 shutdown_connections 2024-08-18T13:39:54.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 --2- 172.21.15.118:0/1344296632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441561d0 0x7fb14415cb00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- 172.21.15.118:0/1344296632 >> 172.21.15.118:0/1344296632 conn(0x7fb14406c020 msgr2=0x7fb1440c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:54.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- 172.21.15.118:0/1344296632 shutdown_connections 2024-08-18T13:39:54.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- 172.21.15.118:0/1344296632 wait complete. 2024-08-18T13:39:54.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 Processor -- start 2024-08-18T13:39:54.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- start start 2024-08-18T13:39:54.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1440684e0 0x7fb1440c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441556f0 0x7fb1440c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1441561d0 0x7fb1440c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440c4e50 con 0x7fb1441556f0 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14415a8c0 con 0x7fb1440684e0 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149d25700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440c50f0 con 0x7fb1441561d0 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441556f0 0x7fb1440c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441556f0 0x7fb1440c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44884/0 (socket says 172.21.15.118:44884) 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 -- 172.21.15.118:0/889011390 learned_addr learned my addr 172.21.15.118:0/889011390 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb148d23700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1440684e0 0x7fb1440c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb149524700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1441561d0 0x7fb1440c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 -- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1441561d0 msgr2=0x7fb1440c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1441561d0 0x7fb1440c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 -- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1440684e0 msgr2=0x7fb1440c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1440684e0 0x7fb1440c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.686+0000 7fb143fff700 1 -- 172.21.15.118:0/889011390 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134009420 con 0x7fb1441556f0 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb143fff700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441556f0 0x7fb1440c66f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00ccc0 tx=0x7fb12c0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb140ff9700 1 -- 172.21.15.118:0/889011390 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00ce70 con 0x7fb1441556f0 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb140ff9700 1 -- 172.21.15.118:0/889011390 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c00ab70 con 0x7fb1441556f0 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb144159b00 con 0x7fb1441556f0 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb140ff9700 1 -- 172.21.15.118:0/889011390 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0070f0 con 0x7fb1441556f0 2024-08-18T13:39:54.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14415a0c0 con 0x7fb1441556f0 2024-08-18T13:39:54.696 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1440713a0 con 0x7fb1441556f0 2024-08-18T13:39:54.696 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb140ff9700 1 -- 172.21.15.118:0/889011390 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb12c0074e0 con 0x7fb1441556f0 2024-08-18T13:39:54.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb140ff9700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb128060860 0x7fb128062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb140ff9700 1 -- 172.21.15.118:0/889011390 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fb12c07d3c0 con 0x7fb1441556f0 2024-08-18T13:39:54.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.690+0000 7fb148d23700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb128060860 0x7fb128062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.694+0000 7fb148d23700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb128060860 0x7fb128062d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb13c003860 tx=0x7fb13c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:54.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.698+0000 7fb140ff9700 1 -- 172.21.15.118:0/889011390 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb12c04d480 con 0x7fb1441556f0 2024-08-18T13:39:54.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.858+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb14414b4f0 con 0x7fb1441556f0 2024-08-18T13:39:54.866 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb140ff9700 1 -- 172.21.15.118:0/889011390 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7fb12c050aa0 con 0x7fb1441556f0 2024-08-18T13:39:54.866 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:54.866 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:54.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb128060860 msgr2=0x7fb128062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb128060860 0x7fb128062d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb13c003860 tx=0x7fb13c009040 comp rx=0 tx=0).stop 2024-08-18T13:39:54.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441556f0 msgr2=0x7fb1440c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441556f0 0x7fb1440c66f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00ccc0 tx=0x7fb12c0019e0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 shutdown_connections 2024-08-18T13:39:54.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb128060860 0x7fb128062d10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1441561d0 0x7fb1440c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb1441556f0 0x7fb1440c66f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 --2- 172.21.15.118:0/889011390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1440684e0 0x7fb1440c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:54.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 >> 172.21.15.118:0/889011390 conn(0x7fb14406c020 msgr2=0x7fb14406e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:54.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 shutdown_connections 2024-08-18T13:39:54.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.862+0000 7fb149d25700 1 -- 172.21.15.118:0/889011390 wait complete. 2024-08-18T13:39:54.884 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-18T13:39:54.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.990+0000 7f8ab736c700 1 Processor -- start 2024-08-18T13:39:54.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.990+0000 7f8ab736c700 1 -- start start 2024-08-18T13:39:54.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8ab006c360 0x7f8ab00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:54.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab00c3550 con 0x7f8ab006c730 2024-08-18T13:39:54.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab0155fb0 con 0x7f8ab00c2830 2024-08-18T13:39:54.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab00696f0 con 0x7f8ab006c360 2024-08-18T13:39:54.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab636a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8ab006c360 0x7f8ab00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:54.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab6b6b700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f8ab006c730 0x7f8ab0152400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45846/0 (socket says 172.21.15.118:45846) 2024-08-18T13:39:54.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab6b6b700 1 -- 172.21.15.118:0/3832871510 learned_addr learned my addr 172.21.15.118:0/3832871510 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:54.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab5368700 1 -- 172.21.15.118:0/3832871510 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256169467 0 0) 0x7f8ab00c3550 con 0x7f8ab006c730 2024-08-18T13:39:54.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab5368700 1 -- 172.21.15.118:0/3832871510 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a9c003570 con 0x7f8ab006c730 2024-08-18T13:39:54.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab636a700 1 -- 172.21.15.118:0/3832871510 >> v1:172.21.15.178:6789/0 conn(0x7f8ab00c2830 legacy=0x7f8ab00c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab636a700 1 -- 172.21.15.118:0/3832871510 >> v1:172.21.15.118:6789/0 conn(0x7f8ab006c730 legacy=0x7f8ab0152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:54.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab636a700 1 -- 172.21.15.118:0/3832871510 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab00c4e70 con 0x7f8ab006c360 2024-08-18T13:39:54.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab636a700 1 --2- 172.21.15.118:0/3832871510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab006c360 0x7f8ab00c8fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00098e0 tx=0x7f8aa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2810ffa847f8862 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:55.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab5368700 1 -- 172.21.15.118:0/3832871510 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000b640 con 0x7f8ab006c360 2024-08-18T13:39:55.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab5368700 1 -- 172.21.15.118:0/3832871510 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa000b7a0 con 0x7f8ab006c360 2024-08-18T13:39:55.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab5368700 1 -- 172.21.15.118:0/3832871510 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000bb40 con 0x7f8ab006c360 2024-08-18T13:39:55.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- 172.21.15.118:0/3832871510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab006c360 msgr2=0x7f8ab00c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 --2- 172.21.15.118:0/3832871510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab006c360 0x7f8ab00c8fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00098e0 tx=0x7f8aa00091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- 172.21.15.118:0/3832871510 shutdown_connections 2024-08-18T13:39:55.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 --2- 172.21.15.118:0/3832871510 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab006c360 0x7f8ab00c8fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- 172.21.15.118:0/3832871510 >> 172.21.15.118:0/3832871510 conn(0x7f8ab006f330 msgr2=0x7f8ab00c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:55.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- 172.21.15.118:0/3832871510 shutdown_connections 2024-08-18T13:39:55.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.994+0000 7f8ab736c700 1 -- 172.21.15.118:0/3832871510 wait complete. 2024-08-18T13:39:55.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 Processor -- start 2024-08-18T13:39:55.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 -- start start 2024-08-18T13:39:55.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ab006c360 0x7f8ab0146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ab006c730 0x7f8ab01664e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab00c2830 0x7f8ab0151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab0153550 con 0x7f8ab00c2830 2024-08-18T13:39:55.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01530c0 con 0x7f8ab006c360 2024-08-18T13:39:55.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab736c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00c47c0 con 0x7f8ab006c730 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab00c2830 0x7f8ab0151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab5b69700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ab006c730 0x7f8ab01664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab5b69700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ab006c730 0x7f8ab01664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54028/0 (socket says 172.21.15.118:54028) 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab00c2830 0x7f8ab0151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44906/0 (socket says 172.21.15.118:44906) 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab5b69700 1 -- 172.21.15.118:0/4249650138 learned_addr learned my addr 172.21.15.118:0/4249650138 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab636a700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ab006c360 0x7f8ab0146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 -- 172.21.15.118:0/4249650138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ab006c730 msgr2=0x7f8ab01664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ab006c730 0x7f8ab01664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 -- 172.21.15.118:0/4249650138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ab006c360 msgr2=0x7f8ab0146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ab006c360 0x7f8ab0146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 -- 172.21.15.118:0/4249650138 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa0009420 con 0x7f8ab00c2830 2024-08-18T13:39:55.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8ab6b6b700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab00c2830 0x7f8ab0151270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800cbf0 tx=0x7f8aa8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:55.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8aa67fc700 1 -- 172.21.15.118:0/4249650138 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800a870 con 0x7f8ab00c2830 2024-08-18T13:39:55.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:54.998+0000 7f8aa67fc700 1 -- 172.21.15.118:0/4249650138 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa800a9d0 con 0x7f8ab00c2830 2024-08-18T13:39:55.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab01693c0 con 0x7f8ab00c2830 2024-08-18T13:39:55.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8aa67fc700 1 -- 172.21.15.118:0/4249650138 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800ad50 con 0x7f8ab00c2830 2024-08-18T13:39:55.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab0152790 con 0x7f8ab00c2830 2024-08-18T13:39:55.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab0071410 con 0x7f8ab00c2830 2024-08-18T13:39:55.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8aa67fc700 1 -- 172.21.15.118:0/4249650138 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8aa8007300 con 0x7f8ab00c2830 2024-08-18T13:39:55.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8aa67fc700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a94060860 0x7f8a94062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8aa67fc700 1 -- 172.21.15.118:0/4249650138 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f8aa807d200 con 0x7f8ab00c2830 2024-08-18T13:39:55.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.002+0000 7f8ab636a700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a94060860 0x7f8a94062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.006+0000 7f8ab636a700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a94060860 0x7f8a94062d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00093f0 tx=0x7f8aa0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:55.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.010+0000 7f8aa67fc700 1 -- 172.21.15.118:0/4249650138 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8aa804d2d0 con 0x7f8ab00c2830 2024-08-18T13:39:55.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.214+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8ab0153200 con 0x7f8ab00c2830 2024-08-18T13:39:55.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.478+0000 7f8aa67fc700 1 -- 172.21.15.118:0/4249650138 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8aa80508f0 con 0x7f8ab00c2830 2024-08-18T13:39:55.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a94060860 msgr2=0x7f8a94062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a94060860 0x7f8a94062d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00093f0 tx=0x7f8aa0017000 comp rx=0 tx=0).stop 2024-08-18T13:39:55.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab00c2830 msgr2=0x7f8ab0151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab00c2830 0x7f8ab0151270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800cbf0 tx=0x7f8aa8001a20 comp rx=0 tx=0).stop 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 shutdown_connections 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a94060860 0x7f8a94062d10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ab00c2830 0x7f8ab0151270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8ab006c730 0x7f8ab01664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 --2- 172.21.15.118:0/4249650138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ab006c360 0x7f8ab0146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 >> 172.21.15.118:0/4249650138 conn(0x7f8ab006f330 msgr2=0x7f8ab00673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 shutdown_connections 2024-08-18T13:39:55.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.482+0000 7f8ab736c700 1 -- 172.21.15.118:0/4249650138 wait complete. 2024-08-18T13:39:55.502 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-18T13:39:55.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.614+0000 7f751cde9700 1 Processor -- start 2024-08-18T13:39:55.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.614+0000 7f751cde9700 1 -- start start 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f751814d220 0x7f75181544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75180c4620 0x7f7518154a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f751806e0b0 0x7f75181565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75180bea10 con 0x7f751814d220 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75180bef40 con 0x7f75180c4620 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75180c0480 con 0x7f751806e0b0 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f751814d220 0x7f75181544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f751814d220 0x7f75181544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44912/0 (socket says 172.21.15.118:44912) 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 -- 172.21.15.118:0/2865746719 learned_addr learned my addr 172.21.15.118:0/2865746719 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f7516ffd700 1 --2- 172.21.15.118:0/2865746719 >> v2:172.21.15.178:3300/0 conn(0x7f75180c4620 0x7f7518154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f7517fff700 1 --2- 172.21.15.118:0/2865746719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f751806e0b0 0x7f75181565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 -- 172.21.15.118:0/2865746719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f751806e0b0 msgr2=0x7f75181565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 --2- 172.21.15.118:0/2865746719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f751806e0b0 0x7f75181565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 -- 172.21.15.118:0/2865746719 >> v2:172.21.15.178:3300/0 conn(0x7f75180c4620 msgr2=0x7f7518154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 --2- 172.21.15.118:0/2865746719 >> v2:172.21.15.178:3300/0 conn(0x7f75180c4620 0x7f7518154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 -- 172.21.15.118:0/2865746719 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180c0760 con 0x7f751814d220 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75177fe700 1 --2- 172.21.15.118:0/2865746719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f75181544c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75180ac980 tx=0x7f7500009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110005adecea0163 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75167fc700 1 -- 172.21.15.118:0/2865746719 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000b6d0 con 0x7f751814d220 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75167fc700 1 -- 172.21.15.118:0/2865746719 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750000b830 con 0x7f751814d220 2024-08-18T13:39:55.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f75167fc700 1 -- 172.21.15.118:0/2865746719 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000bbf0 con 0x7f751814d220 2024-08-18T13:39:55.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- 172.21.15.118:0/2865746719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 msgr2=0x7f75181544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 --2- 172.21.15.118:0/2865746719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f75181544c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75180ac980 tx=0x7f7500009250 comp rx=0 tx=0).stop 2024-08-18T13:39:55.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- 172.21.15.118:0/2865746719 shutdown_connections 2024-08-18T13:39:55.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 --2- 172.21.15.118:0/2865746719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f751806e0b0 0x7f75181565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 --2- 172.21.15.118:0/2865746719 >> v2:172.21.15.178:3300/0 conn(0x7f75180c4620 0x7f7518154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 --2- 172.21.15.118:0/2865746719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f75181544c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- 172.21.15.118:0/2865746719 >> 172.21.15.118:0/2865746719 conn(0x7f751806d5a0 msgr2=0x7f751806d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:55.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- 172.21.15.118:0/2865746719 shutdown_connections 2024-08-18T13:39:55.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.618+0000 7f751cde9700 1 -- 172.21.15.118:0/2865746719 wait complete. 2024-08-18T13:39:55.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 Processor -- start 2024-08-18T13:39:55.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 -- start start 2024-08-18T13:39:55.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f751806e0b0 0x7f7518153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f7518166cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7518165390 0x7f7518165830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75180ad3f0 con 0x7f751814d220 2024-08-18T13:39:55.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518152190 con 0x7f751806e0b0 2024-08-18T13:39:55.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f751cde9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7518152620 con 0x7f7518165390 2024-08-18T13:39:55.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7516ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f7518166cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7516ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f7518166cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44918/0 (socket says 172.21.15.118:44918) 2024-08-18T13:39:55.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7516ffd700 1 -- 172.21.15.118:0/3591748836 learned_addr learned my addr 172.21.15.118:0/3591748836 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:55.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7517fff700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7518165390 0x7f7518165830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f75177fe700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f751806e0b0 0x7f7518153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7517fff700 1 -- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f751806e0b0 msgr2=0x7f7518153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7517fff700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f751806e0b0 0x7f7518153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7517fff700 1 -- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 msgr2=0x7f7518166cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:55.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7517fff700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f7518166cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7517fff700 1 -- 172.21.15.118:0/3591748836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75000094f0 con 0x7f7518165390 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.622+0000 7f7516ffd700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f7518166cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f7517fff700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7518165390 0x7f7518165830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f751000ecf0 tx=0x7f751000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f750f7fe700 1 -- 172.21.15.118:0/3591748836 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751000c980 con 0x7f7518165390 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f750f7fe700 1 -- 172.21.15.118:0/3591748836 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751000f040 con 0x7f7518165390 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751814aaf0 con 0x7f7518165390 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f750f7fe700 1 -- 172.21.15.118:0/3591748836 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751000cd50 con 0x7f7518165390 2024-08-18T13:39:55.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751814ad90 con 0x7f7518165390 2024-08-18T13:39:55.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7518152300 con 0x7f7518165390 2024-08-18T13:39:55.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.626+0000 7f750f7fe700 1 -- 172.21.15.118:0/3591748836 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7510014690 con 0x7f7518165390 2024-08-18T13:39:55.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.630+0000 7f750f7fe700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74f8060860 0x7f74f8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:55.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.630+0000 7f750f7fe700 1 -- 172.21.15.118:0/3591748836 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f751007da10 con 0x7f7518165390 2024-08-18T13:39:55.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.630+0000 7f75177fe700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74f8060860 0x7f74f8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:55.634 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.630+0000 7f75177fe700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74f8060860 0x7f74f8062d10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75180ac980 tx=0x7f7500017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:55.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.642+0000 7f750f7fe700 1 -- 172.21.15.118:0/3591748836 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f751004dad0 con 0x7f7518165390 2024-08-18T13:39:55.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:55.842+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7518151f90 con 0x7f7518165390 2024-08-18T13:39:56.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.486+0000 7f750f7fe700 1 -- 172.21.15.118:0/3591748836 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f75100180c0 con 0x7f7518165390 2024-08-18T13:39:56.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.490+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74f8060860 msgr2=0x7f74f8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:56.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.490+0000 7f751cde9700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74f8060860 0x7f74f8062d10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75180ac980 tx=0x7f7500017000 comp rx=0 tx=0).stop 2024-08-18T13:39:56.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.490+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7518165390 msgr2=0x7f7518165830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:56.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.490+0000 7f751cde9700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7518165390 0x7f7518165830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f751000ecf0 tx=0x7f751000b040 comp rx=0 tx=0).stop 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 shutdown_connections 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74f8060860 0x7f74f8062d10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7518165390 0x7f7518165830 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f751814d220 0x7f7518166cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 --2- 172.21.15.118:0/3591748836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f751806e0b0 0x7f7518153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 >> 172.21.15.118:0/3591748836 conn(0x7f751806d5a0 msgr2=0x7f7518154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 shutdown_connections 2024-08-18T13:39:56.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.494+0000 7f751cde9700 1 -- 172.21.15.118:0/3591748836 wait complete. 2024-08-18T13:39:56.514 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-08-18T13:39:56.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.630+0000 7f1fddb99700 1 Processor -- start 2024-08-18T13:39:56.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.630+0000 7f1fddb99700 1 -- start start 2024-08-18T13:39:56.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.630+0000 7f1fddb99700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1fd80c0cf0 0x7f1fd8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:56.637 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.630+0000 7f1fddb99700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd80c10c0 0x7f1fd80cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80698b0 con 0x7f1fd8069d40 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd8156250 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80cc970 con 0x7f1fd80c10c0 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1fd80c0cf0 0x7f1fd8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fcffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd80c10c0 0x7f1fd80cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1fd80c0cf0 0x7f1fd8155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44934/0 (socket says 172.21.15.118:44934) 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd7fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f1fd8069d40 0x7f1fd814efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:33884/0 (socket says 172.21.15.118:33884) 2024-08-18T13:39:56.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 -- 172.21.15.118:0/403217455 learned_addr learned my addr 172.21.15.118:0/403217455 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:56.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd6ffd700 1 -- 172.21.15.118:0/403217455 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031406216 0 0) 0x7f1fd80698b0 con 0x7f1fd8069d40 2024-08-18T13:39:56.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd6ffd700 1 -- 172.21.15.118:0/403217455 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb8003570 con 0x7f1fd8069d40 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 -- 172.21.15.118:0/403217455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd80c10c0 msgr2=0x7f1fd80cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 --2- 172.21.15.118:0/403217455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd80c10c0 0x7f1fd80cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 -- 172.21.15.118:0/403217455 >> v1:172.21.15.178:6789/0 conn(0x7f1fd8069d40 legacy=0x7f1fd814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 -- 172.21.15.118:0/403217455 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80ccb70 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fcffff700 1 --2- 172.21.15.118:0/403217455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd80c10c0 0x7f1fd80cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd77fe700 1 --2- 172.21.15.118:0/403217455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd8155d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80099e0 tx=0x7f1fc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcb310a16d7bef server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd6ffd700 1 -- 172.21.15.118:0/403217455 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800b6d0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd6ffd700 1 -- 172.21.15.118:0/403217455 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc800b830 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fd6ffd700 1 -- 172.21.15.118:0/403217455 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800bbb0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- 172.21.15.118:0/403217455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 msgr2=0x7f1fd8155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:56.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 --2- 172.21.15.118:0/403217455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd8155d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80099e0 tx=0x7f1fc8009250 comp rx=0 tx=0).stop 2024-08-18T13:39:56.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- 172.21.15.118:0/403217455 shutdown_connections 2024-08-18T13:39:56.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 --2- 172.21.15.118:0/403217455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd80c10c0 0x7f1fd80cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 --2- 172.21.15.118:0/403217455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd8155d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- 172.21.15.118:0/403217455 >> 172.21.15.118:0/403217455 conn(0x7f1fd8067df0 msgr2=0x7f1fd80cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:56.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- 172.21.15.118:0/403217455 shutdown_connections 2024-08-18T13:39:56.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.634+0000 7f1fddb99700 1 -- 172.21.15.118:0/403217455 wait complete. 2024-08-18T13:39:56.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.638+0000 7f1fddb99700 1 Processor -- start 2024-08-18T13:39:56.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.638+0000 7f1fddb99700 1 -- start start 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8069d40 0x7f1fd814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd80b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd80c10c0 0x7f1fd80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80c5300 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80c5e70 con 0x7f1fd80c10c0 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b04d0 con 0x7f1fd8069d40 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd80b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fd77fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8069d40 0x7f1fd814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd80b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44938/0 (socket says 172.21.15.118:44938) 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fd7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd80c10c0 0x7f1fd80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:56.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 -- 172.21.15.118:0/524696817 learned_addr learned my addr 172.21.15.118:0/524696817 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:56.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 -- 172.21.15.118:0/524696817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8069d40 msgr2=0x7f1fd814d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8069d40 0x7f1fd814d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 -- 172.21.15.118:0/524696817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd80c10c0 msgr2=0x7f1fd80b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd80c10c0 0x7f1fd80b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 -- 172.21.15.118:0/524696817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc0009420 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcffff700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd80b0960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000ec80 tx=0x7f1fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcf7fe700 1 -- 172.21.15.118:0/524696817 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000c850 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcf7fe700 1 -- 172.21.15.118:0/524696817 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc000c9b0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc80094f0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fcf7fe700 1 -- 172.21.15.118:0/524696817 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000cd30 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.642+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd80af1b0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.646+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd80642f0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.646+0000 7f1fcf7fe700 1 -- 172.21.15.118:0/524696817 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1fc0013680 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.646+0000 7f1fcf7fe700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1fbc060860 0x7f1fbc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:56.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.646+0000 7f1fd77fe700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1fbc060860 0x7f1fbc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:56.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.646+0000 7f1fcf7fe700 1 -- 172.21.15.118:0/524696817 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f1fc007d0d0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.646+0000 7f1fd77fe700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1fbc060860 0x7f1fbc062d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80099e0 tx=0x7f1fc8017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:56.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.650+0000 7f1fcf7fe700 1 -- 172.21.15.118:0/524696817 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1fc0050cc0 con 0x7f1fd80c0cf0 2024-08-18T13:39:56.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:56.854+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1fd814cc50 con 0x7f1fd80c0cf0 2024-08-18T13:39:57.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.494+0000 7f1fcf7fe700 1 -- 172.21.15.118:0/524696817 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v9) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1fc0050850 con 0x7f1fd80c0cf0 2024-08-18T13:39:57.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1fbc060860 msgr2=0x7f1fbc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1fbc060860 0x7f1fbc062d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80099e0 tx=0x7f1fc8017000 comp rx=0 tx=0).stop 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 msgr2=0x7f1fd80b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd80b0960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000ec80 tx=0x7f1fc00091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 shutdown_connections 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1fbc060860 0x7f1fbc062d10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fd80c10c0 0x7f1fd80b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd80c0cf0 0x7f1fd80b0960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 --2- 172.21.15.118:0/524696817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fd8069d40 0x7f1fd814d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 >> 172.21.15.118:0/524696817 conn(0x7f1fd8067df0 msgr2=0x7f1fd8158180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 shutdown_connections 2024-08-18T13:39:57.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.498+0000 7f1fddb99700 1 -- 172.21.15.118:0/524696817 wait complete. 2024-08-18T13:39:57.522 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:39:57.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0af08b2700 1 Processor -- start 2024-08-18T13:39:57.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0af08b2700 1 -- start start 2024-08-18T13:39:57.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0af08b2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 0x7f0aec159180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0af08b2700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aec0c54d0 con 0x7f0aec0c5960 2024-08-18T13:39:57.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0af08b2700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aec14f940 con 0x7f0aec14f570 2024-08-18T13:39:57.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0af08b2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec0ad2a0 con 0x7f0aec0c6230 2024-08-18T13:39:57.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0ae3fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f0aec0c5960 0x7f0aec06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45852/0 (socket says 172.21.15.118:45852) 2024-08-18T13:39:57.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0ae3fff700 1 -- 172.21.15.118:0/2861648140 learned_addr learned my addr 172.21.15.118:0/2861648140 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:57.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.626+0000 7f0aebfff700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 0x7f0aec159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.638 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947985160 0 0) 0x7f0aec0c54d0 con 0x7f0aec0c5960 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad8003570 con 0x7f0aec0c5960 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4038409317 0 0) 0x7f0ad8003570 con 0x7f0aec0c5960 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 msgr2=0x7f0aec159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 0x7f0aec159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 >> v1:172.21.15.178:6789/0 conn(0x7f0aec14f570 legacy=0x7f0aec0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec0c4600 con 0x7f0aec0c5960 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f0ad4001470 con 0x7f0aec0c5960 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 >> v1:172.21.15.118:6789/0 conn(0x7f0aec0c5960 legacy=0x7f0aec06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ad8003a90 0x7f0ad8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 0x7f0ad8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec0ad2a0 con 0x7f0ad8003a90 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad800bb10 con 0x7f0ad8004d50 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad800bcd0 con 0x7f0aec0c6230 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeb7fe700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ad8003a90 0x7f0ad8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0ae3fff700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 0x7f0ad8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aebfff700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeb7fe700 1 -- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 msgr2=0x7f0ad8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeb7fe700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec0c6230 0x7f0ad8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeb7fe700 1 -- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad8004d50 msgr2=0x7f0ad80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeb7fe700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeb7fe700 1 -- 172.21.15.118:0/2861648140 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8003570 con 0x7f0ad8003a90 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aebfff700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:57.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeb7fe700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ad8003a90 0x7f0ad8003e80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0adc001970 tx=0x7f0adc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d674a34ead4c0e4e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00c5d0 con 0x7f0ad8003a90 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc00c730 con 0x7f0ad8003a90 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aeaffd700 1 -- 172.21.15.118:0/2861648140 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00cad0 con 0x7f0ad8003a90 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0af08b2700 1 -- 172.21.15.118:0/2861648140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ad8003a90 msgr2=0x7f0ad8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0af08b2700 1 --2- 172.21.15.118:0/2861648140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0ad8003a90 0x7f0ad8003e80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0adc001970 tx=0x7f0adc007d10 comp rx=0 tx=0).stop 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0aebfff700 1 -- 172.21.15.118:0/2861648140 reap_dead start 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0af08b2700 1 -- 172.21.15.118:0/2861648140 shutdown_connections 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0af08b2700 1 -- 172.21.15.118:0/2861648140 >> 172.21.15.118:0/2861648140 conn(0x7f0aec0c9c10 msgr2=0x7f0aec0ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0af08b2700 1 -- 172.21.15.118:0/2861648140 shutdown_connections 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.630+0000 7f0af08b2700 1 -- 172.21.15.118:0/2861648140 wait complete. 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 Processor -- start 2024-08-18T13:39:57.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 -- start start 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aec157130 0x7f0aec0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aec157500 0x7f0aec0c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec1523e0 0x7f0aec13cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec158630 con 0x7f0aec157130 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec151e80 con 0x7f0aec157500 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec0c8c10 con 0x7f0aec1523e0 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aec157130 0x7f0aec0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aec157130 0x7f0aec0c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44942/0 (socket says 172.21.15.118:44942) 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 -- 172.21.15.118:0/3220017993 learned_addr learned my addr 172.21.15.118:0/3220017993 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aebfff700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec1523e0 0x7f0aec13cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0ae3fff700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aec157500 0x7f0aec0c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 -- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec1523e0 msgr2=0x7f0aec13cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec1523e0 0x7f0aec13cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 -- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aec157500 msgr2=0x7f0aec0c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aec157500 0x7f0aec0c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 -- 172.21.15.118:0/3220017993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec1580b0 con 0x7f0aec157130 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0ae3fff700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aec157500 0x7f0aec0c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0aeb7fe700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aec157130 0x7f0aec0c90a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0adc012f40 tx=0x7f0adc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0ae37fe700 1 -- 172.21.15.118:0/3220017993 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00c5d0 con 0x7f0aec157130 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0ae37fe700 1 -- 172.21.15.118:0/3220017993 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc00ce40 con 0x7f0aec157130 2024-08-18T13:39:57.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aec158350 con 0x7f0aec157130 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0ae37fe700 1 -- 172.21.15.118:0/3220017993 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc007af0 con 0x7f0aec157130 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.634+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aec0c88a0 con 0x7f0aec157130 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.638+0000 7f0ae37fe700 1 -- 172.21.15.118:0/3220017993 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0adc019030 con 0x7f0aec157130 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.638+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aec0c8420 con 0x7f0aec157130 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.638+0000 7f0ae37fe700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0acc060890 0x7f0acc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.638+0000 7f0ae37fe700 1 -- 172.21.15.118:0/3220017993 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f0adc07e050 con 0x7f0aec157130 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.638+0000 7f0ae3fff700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0acc060890 0x7f0acc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.638+0000 7f0ae3fff700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0acc060890 0x7f0acc062d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40015a0 tx=0x7f0ad4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:57.649 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.646+0000 7f0ae37fe700 1 -- 172.21.15.118:0/3220017993 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0aec0c8420 con 0x7f0aec157130 2024-08-18T13:39:57.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.850+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0aec0c8420 con 0x7f0aec157130 2024-08-18T13:39:57.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.850+0000 7f0ae37fe700 1 -- 172.21.15.118:0/3220017993 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7f0aec0c8420 con 0x7f0aec157130 2024-08-18T13:39:57.856 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:57.856 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:39:57.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0acc060890 msgr2=0x7f0acc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0acc060890 0x7f0acc062d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40015a0 tx=0x7f0ad4009040 comp rx=0 tx=0).stop 2024-08-18T13:39:57.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aec157130 msgr2=0x7f0aec0c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aec157130 0x7f0aec0c90a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0adc012f40 tx=0x7f0adc015000 comp rx=0 tx=0).stop 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 shutdown_connections 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0acc060890 0x7f0acc062d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0aec1523e0 0x7f0aec13cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aec157500 0x7f0aec0c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 --2- 172.21.15.118:0/3220017993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aec157130 0x7f0aec0c90a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 >> 172.21.15.118:0/3220017993 conn(0x7f0aec0c9c10 msgr2=0x7f0aec14e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 shutdown_connections 2024-08-18T13:39:57.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.854+0000 7f0af08b2700 1 -- 172.21.15.118:0/3220017993 wait complete. 2024-08-18T13:39:57.860 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:39:57.871 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:57.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb27556c700 1 Processor -- start 2024-08-18T13:39:57.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb27556c700 1 -- start start 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb27556c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb270153690 0x7fb2700673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb27556c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb2700c8200 0x7fb27006f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb27556c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27006ee60 con 0x7fb2700c22a0 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb27556c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27015a640 con 0x7fb2700c8200 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb27556c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb270153d10 con 0x7fb270153690 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb26ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb270153690 0x7fb2700673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.970+0000 7fb26f7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb2700c8200 0x7fb27006f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb270153690 0x7fb2700673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54096/0 (socket says 172.21.15.118:54096) 2024-08-18T13:39:57.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26f7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb2700c8200 0x7fb27006f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:43718/0 (socket says 172.21.15.118:43718) 2024-08-18T13:39:57.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26ffff700 1 -- 172.21.15.118:0/2158851720 learned_addr learned my addr 172.21.15.118:0/2158851720 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26f7fe700 1 -- 172.21.15.118:0/2158851720 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb270153690 msgr2=0x7fb2700673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26f7fe700 1 --2- 172.21.15.118:0/2158851720 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb270153690 0x7fb2700673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26f7fe700 1 -- 172.21.15.118:0/2158851720 >> v1:172.21.15.178:6789/0 conn(0x7fb2700c22a0 legacy=0x7fb2700c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26f7fe700 1 -- 172.21.15.118:0/2158851720 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27006f9c0 con 0x7fb2700c8200 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26f7fe700 1 --2- 172.21.15.118:0/2158851720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2700c8200 0x7fb27006f2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb258009e60 tx=0x7fb258009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=456c6bb78c77575a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26effd700 1 -- 172.21.15.118:0/2158851720 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800e070 con 0x7fb2700c8200 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26effd700 1 -- 172.21.15.118:0/2158851720 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25800bb60 con 0x7fb2700c8200 2024-08-18T13:39:57.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb26effd700 1 -- 172.21.15.118:0/2158851720 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258012430 con 0x7fb2700c8200 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 -- 172.21.15.118:0/2158851720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2700c8200 msgr2=0x7fb27006f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 --2- 172.21.15.118:0/2158851720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2700c8200 0x7fb27006f2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb258009e60 tx=0x7fb258009780 comp rx=0 tx=0).stop 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 -- 172.21.15.118:0/2158851720 shutdown_connections 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 --2- 172.21.15.118:0/2158851720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2700c8200 0x7fb27006f2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 --2- 172.21.15.118:0/2158851720 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb270153690 0x7fb2700673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 -- 172.21.15.118:0/2158851720 >> 172.21.15.118:0/2158851720 conn(0x7fb27006c020 msgr2=0x7fb2700c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 -- 172.21.15.118:0/2158851720 shutdown_connections 2024-08-18T13:39:57.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.974+0000 7fb27556c700 1 -- 172.21.15.118:0/2158851720 wait complete. 2024-08-18T13:39:57.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 Processor -- start 2024-08-18T13:39:57.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 -- start start 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2700c22a0 0x7fb2700c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb2700c8200 0x7fb2700c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb270153690 0x7fb270169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270157080 con 0x7fb2700c8200 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb270156470 con 0x7fb270153690 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb270156750 con 0x7fb2700c22a0 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb2700c8200 0x7fb2700c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb2700c8200 0x7fb2700c01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44954/0 (socket says 172.21.15.118:44954) 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 -- 172.21.15.118:0/2470128280 learned_addr learned my addr 172.21.15.118:0/2470128280 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26ffff700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2700c22a0 0x7fb2700c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb274d6b700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb270153690 0x7fb270169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 -- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2700c22a0 msgr2=0x7fb2700c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2700c22a0 0x7fb2700c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 -- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb270153690 msgr2=0x7fb270169e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb270153690 0x7fb270169e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 -- 172.21.15.118:0/2470128280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258009a20 con 0x7fb2700c8200 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26ffff700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2700c22a0 0x7fb2700c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb26f7fe700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb2700c8200 0x7fb2700c01b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb27005eb30 tx=0x7fb258019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb267fff700 1 -- 172.21.15.118:0/2470128280 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258012cc0 con 0x7fb2700c8200 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb270156a30 con 0x7fb2700c8200 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb267fff700 1 -- 172.21.15.118:0/2470128280 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258012e40 con 0x7fb2700c8200 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2700c5790 con 0x7fb2700c8200 2024-08-18T13:39:57.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.978+0000 7fb267fff700 1 -- 172.21.15.118:0/2470128280 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580276c0 con 0x7fb2700c8200 2024-08-18T13:39:57.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.982+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27005ecf0 con 0x7fb2700c8200 2024-08-18T13:39:57.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.982+0000 7fb267fff700 1 -- 172.21.15.118:0/2470128280 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb258024070 con 0x7fb2700c8200 2024-08-18T13:39:57.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.982+0000 7fb267fff700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb250060890 0x7fb250062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:57.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.982+0000 7fb267fff700 1 -- 172.21.15.118:0/2470128280 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fb25807f550 con 0x7fb2700c8200 2024-08-18T13:39:57.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.982+0000 7fb26ffff700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb250060890 0x7fb250062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:57.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.986+0000 7fb26ffff700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb250060890 0x7fb250062d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260009250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:57.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:57.990+0000 7fb267fff700 1 -- 172.21.15.118:0/2470128280 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb27005ecf0 con 0x7fb2700c8200 2024-08-18T13:39:58.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.154+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2700c2000 con 0x7fb2700c8200 2024-08-18T13:39:58.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.154+0000 7fb267fff700 1 -- 172.21.15.118:0/2470128280 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7fb2700c2000 con 0x7fb2700c8200 2024-08-18T13:39:58.161 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:58.161 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb250060890 msgr2=0x7fb250062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb250060890 0x7fb250062d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb260006fd0 tx=0x7fb260009250 comp rx=0 tx=0).stop 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb2700c8200 msgr2=0x7fb2700c01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb2700c8200 0x7fb2700c01b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb27005eb30 tx=0x7fb258019000 comp rx=0 tx=0).stop 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 shutdown_connections 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb250060890 0x7fb250062d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb270153690 0x7fb270169e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb2700c8200 0x7fb2700c01b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 --2- 172.21.15.118:0/2470128280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2700c22a0 0x7fb2700c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 >> 172.21.15.118:0/2470128280 conn(0x7fb27006c020 msgr2=0x7fb2700c2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:58.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 shutdown_connections 2024-08-18T13:39:58.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.158+0000 7fb27556c700 1 -- 172.21.15.118:0/2470128280 wait complete. 2024-08-18T13:39:58.175 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-18T13:39:58.175 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:39:58.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f166d37d700 1 Processor -- start 2024-08-18T13:39:58.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f166d37d700 1 -- start start 2024-08-18T13:39:58.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f166d37d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1668158720 0x7f16680adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f166d37d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16680bfe20 con 0x7f166814f330 2024-08-18T13:39:58.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f166d37d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16680ad3f0 con 0x7f16680c02b0 2024-08-18T13:39:58.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f166d37d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16680bef50 con 0x7f1668158720 2024-08-18T13:39:58.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f16677fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1668158720 0x7f16680adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f16677fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1668158720 0x7f16680adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44956/0 (socket says 172.21.15.118:44956) 2024-08-18T13:39:58.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.278+0000 7f16677fe700 1 -- 172.21.15.118:0/371043850 learned_addr learned my addr 172.21.15.118:0/371043850 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:58.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f16677fe700 1 -- 172.21.15.118:0/371043850 >> v1:172.21.15.178:6789/0 conn(0x7f16680c02b0 legacy=0x7f166815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:58.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f16677fe700 1 -- 172.21.15.118:0/371043850 >> v1:172.21.15.118:6789/0 conn(0x7f166814f330 legacy=0x7f166806bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f16677fe700 1 -- 172.21.15.118:0/371043850 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680bf210 con 0x7f1668158720 2024-08-18T13:39:58.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f16677fe700 1 --2- 172.21.15.118:0/371043850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f16680adaf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1650009f20 tx=0x7f1650009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3b0c917e82e7e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f1666ffd700 1 -- 172.21.15.118:0/371043850 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165000e070 con 0x7f1668158720 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f1666ffd700 1 -- 172.21.15.118:0/371043850 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165000bad0 con 0x7f1668158720 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f1666ffd700 1 -- 172.21.15.118:0/371043850 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500092e0 con 0x7f1668158720 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 -- 172.21.15.118:0/371043850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 msgr2=0x7f16680adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 --2- 172.21.15.118:0/371043850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f16680adaf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1650009f20 tx=0x7f1650009840 comp rx=0 tx=0).stop 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 -- 172.21.15.118:0/371043850 shutdown_connections 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 --2- 172.21.15.118:0/371043850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f16680adaf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 -- 172.21.15.118:0/371043850 >> 172.21.15.118:0/371043850 conn(0x7f16680ca6d0 msgr2=0x7f166806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 -- 172.21.15.118:0/371043850 shutdown_connections 2024-08-18T13:39:58.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 -- 172.21.15.118:0/371043850 wait complete. 2024-08-18T13:39:58.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 Processor -- start 2024-08-18T13:39:58.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.282+0000 7f166d37d700 1 -- start start 2024-08-18T13:39:58.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16680c02b0 0x7f16680aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f166814f330 0x7f1668147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f1668147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1668166260 con 0x7f1668158720 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681695e0 con 0x7f166814f330 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1668168a10 con 0x7f16680c02b0 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f1668147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f1668147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44966/0 (socket says 172.21.15.118:44966) 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 -- 172.21.15.118:0/154898554 learned_addr learned my addr 172.21.15.118:0/154898554 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f16677fe700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f166814f330 0x7f1668147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f1667fff700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16680c02b0 0x7f16680aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 -- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16680c02b0 msgr2=0x7f16680aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16680c02b0 0x7f16680aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 -- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f166814f330 msgr2=0x7f1668147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f166814f330 0x7f1668147550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 -- 172.21.15.118:0/154898554 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1650009ae0 con 0x7f1668158720 2024-08-18T13:39:58.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166cb7c700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f1668147a90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f166000c5e0 tx=0x7f1660007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f165ffff700 1 -- 172.21.15.118:0/154898554 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660011590 con 0x7f1668158720 2024-08-18T13:39:58.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1668168cb0 con 0x7f1668158720 2024-08-18T13:39:58.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f165ffff700 1 -- 172.21.15.118:0/154898554 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660011710 con 0x7f1668158720 2024-08-18T13:39:58.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1668169270 con 0x7f1668158720 2024-08-18T13:39:58.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f165ffff700 1 -- 172.21.15.118:0/154898554 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660002140 con 0x7f1668158720 2024-08-18T13:39:58.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.286+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16680713a0 con 0x7f1668158720 2024-08-18T13:39:58.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.290+0000 7f165ffff700 1 -- 172.21.15.118:0/154898554 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1660011a90 con 0x7f1668158720 2024-08-18T13:39:58.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.290+0000 7f165ffff700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f164c060860 0x7f164c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.290+0000 7f165ffff700 1 -- 172.21.15.118:0/154898554 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f166007c970 con 0x7f1668158720 2024-08-18T13:39:58.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.290+0000 7f1667fff700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f164c060860 0x7f164c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.290+0000 7f1667fff700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f164c060860 0x7f164c062d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1658003a80 tx=0x7f1658007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.294+0000 7f165ffff700 1 -- 172.21.15.118:0/154898554 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f166004ca30 con 0x7f1668158720 2024-08-18T13:39:58.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.454+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1668169720 con 0x7f164c060860 2024-08-18T13:39:58.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.478+0000 7f165ffff700 1 -- 172.21.15.118:0/154898554 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345217 (secure 0 0 0) 0x7f1668169720 con 0x7f164c060860 2024-08-18T13:39:58.486 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:58.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f164c060860 msgr2=0x7f164c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f164c060860 0x7f164c062d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1658003a80 tx=0x7f1658007ec0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 msgr2=0x7f1668147a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f1668147a90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f166000c5e0 tx=0x7f1660007f60 comp rx=0 tx=0).stop 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 shutdown_connections 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f164c060860 0x7f164c062d10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1668158720 0x7f1668147a90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f166814f330 0x7f1668147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 --2- 172.21.15.118:0/154898554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16680c02b0 0x7f16680aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 >> 172.21.15.118:0/154898554 conn(0x7f16680ca6d0 msgr2=0x7f16680c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 shutdown_connections 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.482+0000 7f166d37d700 1 -- 172.21.15.118:0/154898554 wait complete. 2024-08-18T13:39:58.490 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:39:58.501 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-08-18T13:39:57.203240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594916,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415368,"kb_used_data":3240,"kb_used_omap":13,"kb_used_meta":2412018,"kb_avail":752559352,"statfs":{"total":773094113280,"available":770620776448,"internally_reserved":0,"allocated":3317760,"data_stored":2006336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2469907136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2210,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004938"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433629+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:39:40.433629+0000","last_peered":"2024-08-18T13:39:40.433629+0000","last_clean":"2024-08-18T13:39:40.433629+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:39:40.433629+0000","last_undegraded":"2024-08-18T13:39:40.433629+0000","last_fullsized":"2024-08-18T13:39:40.433629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422872+0000","last_change":"2024-08-18T13:39:35.405966+0000","last_active":"2024-08-18T13:39:39.422872+0000","last_peered":"2024-08-18T13:39:39.422872+0000","last_clean":"2024-08-18T13:39:39.422872+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:39:39.422872+0000","last_undegraded":"2024-08-18T13:39:39.422872+0000","last_fullsized":"2024-08-18T13:39:39.422872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454715+0000","last_change":"2024-08-18T13:39:39.453674+0000","last_active":"2024-08-18T13:39:39.454715+0000","last_peered":"2024-08-18T13:39:39.454715+0000","last_clean":"2024-08-18T13:39:39.454715+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:39:39.454715+0000","last_undegraded":"2024-08-18T13:39:39.454715+0000","last_fullsized":"2024-08-18T13:39:39.454715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434410+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:39:40.434410+0000","last_peered":"2024-08-18T13:39:40.434410+0000","last_clean":"2024-08-18T13:39:40.434410+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:39:40.434410+0000","last_undegraded":"2024-08-18T13:39:40.434410+0000","last_fullsized":"2024-08-18T13:39:40.434410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432862+0000","last_change":"2024-08-18T13:39:35.408917+0000","last_active":"2024-08-18T13:39:39.432862+0000","last_peered":"2024-08-18T13:39:39.432862+0000","last_clean":"2024-08-18T13:39:39.432862+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:39:39.432862+0000","last_undegraded":"2024-08-18T13:39:39.432862+0000","last_fullsized":"2024-08-18T13:39:39.432862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447766+0000","last_change":"2024-08-18T13:39:39.445388+0000","last_active":"2024-08-18T13:39:39.447766+0000","last_peered":"2024-08-18T13:39:39.447766+0000","last_clean":"2024-08-18T13:39:39.447766+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:39:39.447766+0000","last_undegraded":"2024-08-18T13:39:39.447766+0000","last_fullsized":"2024-08-18T13:39:39.447766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430049+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:39:40.430049+0000","last_peered":"2024-08-18T13:39:40.430049+0000","last_clean":"2024-08-18T13:39:40.430049+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:39:40.430049+0000","last_undegraded":"2024-08-18T13:39:40.430049+0000","last_fullsized":"2024-08-18T13:39:40.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422649+0000","last_change":"2024-08-18T13:39:35.399188+0000","last_active":"2024-08-18T13:39:39.422649+0000","last_peered":"2024-08-18T13:39:39.422649+0000","last_clean":"2024-08-18T13:39:39.422649+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:39:39.422649+0000","last_undegraded":"2024-08-18T13:39:39.422649+0000","last_fullsized":"2024-08-18T13:39:39.422649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447285+0000","last_change":"2024-08-18T13:39:39.440922+0000","last_active":"2024-08-18T13:39:39.447285+0000","last_peered":"2024-08-18T13:39:39.447285+0000","last_clean":"2024-08-18T13:39:39.447285+0000","last_became_active":"2024-08-18T13:39:39.440426+0000","last_became_peered":"2024-08-18T13:39:39.440426+0000","last_unstale":"2024-08-18T13:39:39.447285+0000","last_undegraded":"2024-08-18T13:39:39.447285+0000","last_fullsized":"2024-08-18T13:39:39.447285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:23:00.376805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.428764+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:39:40.428764+0000","last_peered":"2024-08-18T13:39:40.428764+0000","last_clean":"2024-08-18T13:39:40.428764+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:39:40.428764+0000","last_undegraded":"2024-08-18T13:39:40.428764+0000","last_fullsized":"2024-08-18T13:39:40.428764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423184+0000","last_change":"2024-08-18T13:39:35.400038+0000","last_active":"2024-08-18T13:39:39.423184+0000","last_peered":"2024-08-18T13:39:39.423184+0000","last_clean":"2024-08-18T13:39:39.423184+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:39:39.423184+0000","last_undegraded":"2024-08-18T13:39:39.423184+0000","last_fullsized":"2024-08-18T13:39:39.423184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437377+0000","last_change":"2024-08-18T13:39:39.436221+0000","last_active":"2024-08-18T13:39:39.437377+0000","last_peered":"2024-08-18T13:39:39.437377+0000","last_clean":"2024-08-18T13:39:39.437377+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:39:39.437377+0000","last_undegraded":"2024-08-18T13:39:39.437377+0000","last_fullsized":"2024-08-18T13:39:39.437377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428676+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.428676+0000","last_peered":"2024-08-18T13:39:39.428676+0000","last_clean":"2024-08-18T13:39:39.428676+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:39:39.428676+0000","last_undegraded":"2024-08-18T13:39:39.428676+0000","last_fullsized":"2024-08-18T13:39:39.428676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430090+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:39:40.430090+0000","last_peered":"2024-08-18T13:39:40.430090+0000","last_clean":"2024-08-18T13:39:40.430090+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:39:40.430090+0000","last_undegraded":"2024-08-18T13:39:40.430090+0000","last_fullsized":"2024-08-18T13:39:40.430090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.443080+0000","last_change":"2024-08-18T13:39:39.439633+0000","last_active":"2024-08-18T13:39:39.443080+0000","last_peered":"2024-08-18T13:39:39.443080+0000","last_clean":"2024-08-18T13:39:39.443080+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:39:39.443080+0000","last_undegraded":"2024-08-18T13:39:39.443080+0000","last_fullsized":"2024-08-18T13:39:39.443080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423607+0000","last_change":"2024-08-18T13:39:35.407947+0000","last_active":"2024-08-18T13:39:39.423607+0000","last_peered":"2024-08-18T13:39:39.423607+0000","last_clean":"2024-08-18T13:39:39.423607+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:39:39.423607+0000","last_undegraded":"2024-08-18T13:39:39.423607+0000","last_fullsized":"2024-08-18T13:39:39.423607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429156+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:39:39.429156+0000","last_peered":"2024-08-18T13:39:39.429156+0000","last_clean":"2024-08-18T13:39:39.429156+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:39:39.429156+0000","last_undegraded":"2024-08-18T13:39:39.429156+0000","last_fullsized":"2024-08-18T13:39:39.429156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442892+0000","last_change":"2024-08-18T13:39:39.439913+0000","last_active":"2024-08-18T13:39:39.442892+0000","last_peered":"2024-08-18T13:39:39.442892+0000","last_clean":"2024-08-18T13:39:39.442892+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:39:39.442892+0000","last_undegraded":"2024-08-18T13:39:39.442892+0000","last_fullsized":"2024-08-18T13:39:39.442892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430332+0000","last_change":"2024-08-18T13:39:35.403254+0000","last_active":"2024-08-18T13:39:39.430332+0000","last_peered":"2024-08-18T13:39:39.430332+0000","last_clean":"2024-08-18T13:39:39.430332+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:39:39.430332+0000","last_undegraded":"2024-08-18T13:39:39.430332+0000","last_fullsized":"2024-08-18T13:39:39.430332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422689+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:39:39.422689+0000","last_peered":"2024-08-18T13:39:39.422689+0000","last_clean":"2024-08-18T13:39:39.422689+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:39:39.422689+0000","last_undegraded":"2024-08-18T13:39:39.422689+0000","last_fullsized":"2024-08-18T13:39:39.422689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457822+0000","last_change":"2024-08-18T13:39:39.457250+0000","last_active":"2024-08-18T13:39:39.457822+0000","last_peered":"2024-08-18T13:39:39.457822+0000","last_clean":"2024-08-18T13:39:39.457822+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:39:39.457822+0000","last_undegraded":"2024-08-18T13:39:39.457822+0000","last_fullsized":"2024-08-18T13:39:39.457822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425047+0000","last_change":"2024-08-18T13:39:35.401612+0000","last_active":"2024-08-18T13:39:39.425047+0000","last_peered":"2024-08-18T13:39:39.425047+0000","last_clean":"2024-08-18T13:39:39.425047+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:39:39.425047+0000","last_undegraded":"2024-08-18T13:39:39.425047+0000","last_fullsized":"2024-08-18T13:39:39.425047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450234+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:39:39.450234+0000","last_peered":"2024-08-18T13:39:39.450234+0000","last_clean":"2024-08-18T13:39:39.450234+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:39:39.450234+0000","last_undegraded":"2024-08-18T13:39:39.450234+0000","last_fullsized":"2024-08-18T13:39:39.450234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447688+0000","last_change":"2024-08-18T13:39:39.443503+0000","last_active":"2024-08-18T13:39:39.447688+0000","last_peered":"2024-08-18T13:39:39.447688+0000","last_clean":"2024-08-18T13:39:39.447688+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:39:39.447688+0000","last_undegraded":"2024-08-18T13:39:39.447688+0000","last_fullsized":"2024-08-18T13:39:39.447688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432976+0000","last_change":"2024-08-18T13:39:35.407907+0000","last_active":"2024-08-18T13:39:39.432976+0000","last_peered":"2024-08-18T13:39:39.432976+0000","last_clean":"2024-08-18T13:39:39.432976+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:39:39.432976+0000","last_undegraded":"2024-08-18T13:39:39.432976+0000","last_fullsized":"2024-08-18T13:39:39.432976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429181+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:39:40.429181+0000","last_peered":"2024-08-18T13:39:40.429181+0000","last_clean":"2024-08-18T13:39:40.429181+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:39:40.429181+0000","last_undegraded":"2024-08-18T13:39:40.429181+0000","last_fullsized":"2024-08-18T13:39:40.429181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454767+0000","last_change":"2024-08-18T13:39:39.453138+0000","last_active":"2024-08-18T13:39:39.454767+0000","last_peered":"2024-08-18T13:39:39.454767+0000","last_clean":"2024-08-18T13:39:39.454767+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:39:39.454767+0000","last_undegraded":"2024-08-18T13:39:39.454767+0000","last_fullsized":"2024-08-18T13:39:39.454767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424552+0000","last_change":"2024-08-18T13:39:35.404453+0000","last_active":"2024-08-18T13:39:39.424552+0000","last_peered":"2024-08-18T13:39:39.424552+0000","last_clean":"2024-08-18T13:39:39.424552+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:39:39.424552+0000","last_undegraded":"2024-08-18T13:39:39.424552+0000","last_fullsized":"2024-08-18T13:39:39.424552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:41.051281+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:39:41.051281+0000","last_peered":"2024-08-18T13:39:41.051281+0000","last_clean":"2024-08-18T13:39:41.051281+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:39:41.051281+0000","last_undegraded":"2024-08-18T13:39:41.051281+0000","last_fullsized":"2024-08-18T13:39:41.051281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450645+0000","last_change":"2024-08-18T13:39:39.449851+0000","last_active":"2024-08-18T13:39:39.450645+0000","last_peered":"2024-08-18T13:39:39.450645+0000","last_clean":"2024-08-18T13:39:39.450645+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:39:39.450645+0000","last_undegraded":"2024-08-18T13:39:39.450645+0000","last_fullsized":"2024-08-18T13:39:39.450645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424459+0000","last_change":"2024-08-18T13:39:35.400604+0000","last_active":"2024-08-18T13:39:39.424459+0000","last_peered":"2024-08-18T13:39:39.424459+0000","last_clean":"2024-08-18T13:39:39.424459+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:39:39.424459+0000","last_undegraded":"2024-08-18T13:39:39.424459+0000","last_fullsized":"2024-08-18T13:39:39.424459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450871+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:39:39.450871+0000","last_peered":"2024-08-18T13:39:39.450871+0000","last_clean":"2024-08-18T13:39:39.450871+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:39:39.450871+0000","last_undegraded":"2024-08-18T13:39:39.450871+0000","last_fullsized":"2024-08-18T13:39:39.450871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454096+0000","last_change":"2024-08-18T13:39:39.453599+0000","last_active":"2024-08-18T13:39:39.454096+0000","last_peered":"2024-08-18T13:39:39.454096+0000","last_clean":"2024-08-18T13:39:39.454096+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:39:39.454096+0000","last_undegraded":"2024-08-18T13:39:39.454096+0000","last_fullsized":"2024-08-18T13:39:39.454096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423160+0000","last_change":"2024-08-18T13:39:35.410167+0000","last_active":"2024-08-18T13:39:39.423160+0000","last_peered":"2024-08-18T13:39:39.423160+0000","last_clean":"2024-08-18T13:39:39.423160+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:39:39.423160+0000","last_undegraded":"2024-08-18T13:39:39.423160+0000","last_fullsized":"2024-08-18T13:39:39.423160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424551+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:39:39.424551+0000","last_peered":"2024-08-18T13:39:39.424551+0000","last_clean":"2024-08-18T13:39:39.424551+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:39:39.424551+0000","last_undegraded":"2024-08-18T13:39:39.424551+0000","last_fullsized":"2024-08-18T13:39:39.424551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444517+0000","last_change":"2024-08-18T13:39:39.439948+0000","last_active":"2024-08-18T13:39:39.444517+0000","last_peered":"2024-08-18T13:39:39.444517+0000","last_clean":"2024-08-18T13:39:39.444517+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:39:39.444517+0000","last_undegraded":"2024-08-18T13:39:39.444517+0000","last_fullsized":"2024-08-18T13:39:39.444517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422894+0000","last_change":"2024-08-18T13:39:35.407829+0000","last_active":"2024-08-18T13:39:39.422894+0000","last_peered":"2024-08-18T13:39:39.422894+0000","last_clean":"2024-08-18T13:39:39.422894+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:39:39.422894+0000","last_undegraded":"2024-08-18T13:39:39.422894+0000","last_fullsized":"2024-08-18T13:39:39.422894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417877+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:39:39.417877+0000","last_peered":"2024-08-18T13:39:39.417877+0000","last_clean":"2024-08-18T13:39:39.417877+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:39:39.417877+0000","last_undegraded":"2024-08-18T13:39:39.417877+0000","last_fullsized":"2024-08-18T13:39:39.417877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445479+0000","last_change":"2024-08-18T13:39:39.442372+0000","last_active":"2024-08-18T13:39:39.445479+0000","last_peered":"2024-08-18T13:39:39.445479+0000","last_clean":"2024-08-18T13:39:39.445479+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:39:39.445479+0000","last_undegraded":"2024-08-18T13:39:39.445479+0000","last_fullsized":"2024-08-18T13:39:39.445479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421862+0000","last_change":"2024-08-18T13:39:35.400384+0000","last_active":"2024-08-18T13:39:39.421862+0000","last_peered":"2024-08-18T13:39:39.421862+0000","last_clean":"2024-08-18T13:39:39.421862+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:39:39.421862+0000","last_undegraded":"2024-08-18T13:39:39.421862+0000","last_fullsized":"2024-08-18T13:39:39.421862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415832+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:39:39.415832+0000","last_peered":"2024-08-18T13:39:39.415832+0000","last_clean":"2024-08-18T13:39:39.415832+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:39:39.415832+0000","last_undegraded":"2024-08-18T13:39:39.415832+0000","last_fullsized":"2024-08-18T13:39:39.415832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446859+0000","last_change":"2024-08-18T13:39:39.446401+0000","last_active":"2024-08-18T13:39:39.446859+0000","last_peered":"2024-08-18T13:39:39.446859+0000","last_clean":"2024-08-18T13:39:39.446859+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:39:39.446859+0000","last_undegraded":"2024-08-18T13:39:39.446859+0000","last_fullsized":"2024-08-18T13:39:39.446859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424307+0000","last_change":"2024-08-18T13:39:35.410127+0000","last_active":"2024-08-18T13:39:39.424307+0000","last_peered":"2024-08-18T13:39:39.424307+0000","last_clean":"2024-08-18T13:39:39.424307+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:39:39.424307+0000","last_undegraded":"2024-08-18T13:39:39.424307+0000","last_fullsized":"2024-08-18T13:39:39.424307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450338+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:39:39.450338+0000","last_peered":"2024-08-18T13:39:39.450338+0000","last_clean":"2024-08-18T13:39:39.450338+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:39:39.450338+0000","last_undegraded":"2024-08-18T13:39:39.450338+0000","last_fullsized":"2024-08-18T13:39:39.450338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454861+0000","last_change":"2024-08-18T13:39:39.453901+0000","last_active":"2024-08-18T13:39:39.454861+0000","last_peered":"2024-08-18T13:39:39.454861+0000","last_clean":"2024-08-18T13:39:39.454861+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:39:39.454861+0000","last_undegraded":"2024-08-18T13:39:39.454861+0000","last_fullsized":"2024-08-18T13:39:39.454861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430338+0000","last_change":"2024-08-18T13:39:35.412156+0000","last_active":"2024-08-18T13:39:39.430338+0000","last_peered":"2024-08-18T13:39:39.430338+0000","last_clean":"2024-08-18T13:39:39.430338+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:39:39.430338+0000","last_undegraded":"2024-08-18T13:39:39.430338+0000","last_fullsized":"2024-08-18T13:39:39.430338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417339+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:39:39.417339+0000","last_peered":"2024-08-18T13:39:39.417339+0000","last_clean":"2024-08-18T13:39:39.417339+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:39:39.417339+0000","last_undegraded":"2024-08-18T13:39:39.417339+0000","last_fullsized":"2024-08-18T13:39:39.417339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450727+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:39:39.450727+0000","last_peered":"2024-08-18T13:39:39.450727+0000","last_clean":"2024-08-18T13:39:39.450727+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:39:39.450727+0000","last_undegraded":"2024-08-18T13:39:39.450727+0000","last_fullsized":"2024-08-18T13:39:39.450727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454332+0000","last_change":"2024-08-18T13:39:39.453360+0000","last_active":"2024-08-18T13:39:39.454332+0000","last_peered":"2024-08-18T13:39:39.454332+0000","last_clean":"2024-08-18T13:39:39.454332+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:39:39.454332+0000","last_undegraded":"2024-08-18T13:39:39.454332+0000","last_fullsized":"2024-08-18T13:39:39.454332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450556+0000","last_change":"2024-08-18T13:39:35.421765+0000","last_active":"2024-08-18T13:39:39.450556+0000","last_peered":"2024-08-18T13:39:39.450556+0000","last_clean":"2024-08-18T13:39:39.450556+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:39:39.450556+0000","last_undegraded":"2024-08-18T13:39:39.450556+0000","last_fullsized":"2024-08-18T13:39:39.450556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416404+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:39:39.416404+0000","last_peered":"2024-08-18T13:39:39.416404+0000","last_clean":"2024-08-18T13:39:39.416404+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:39:39.416404+0000","last_undegraded":"2024-08-18T13:39:39.416404+0000","last_fullsized":"2024-08-18T13:39:39.416404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450222+0000","last_change":"2024-08-18T13:39:39.449653+0000","last_active":"2024-08-18T13:39:39.450222+0000","last_peered":"2024-08-18T13:39:39.450222+0000","last_clean":"2024-08-18T13:39:39.450222+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:39:39.450222+0000","last_undegraded":"2024-08-18T13:39:39.450222+0000","last_fullsized":"2024-08-18T13:39:39.450222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422710+0000","last_change":"2024-08-18T13:39:35.407439+0000","last_active":"2024-08-18T13:39:39.422710+0000","last_peered":"2024-08-18T13:39:39.422710+0000","last_clean":"2024-08-18T13:39:39.422710+0000","last_became_active":"2024-08-18T13:39:35.407128+0000","last_became_peered":"2024-08-18T13:39:35.407128+0000","last_unstale":"2024-08-18T13:39:39.422710+0000","last_undegraded":"2024-08-18T13:39:39.422710+0000","last_fullsized":"2024-08-18T13:39:39.422710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431851+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:39:40.431851+0000","last_peered":"2024-08-18T13:39:40.431851+0000","last_clean":"2024-08-18T13:39:40.431851+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:39:40.431851+0000","last_undegraded":"2024-08-18T13:39:40.431851+0000","last_fullsized":"2024-08-18T13:39:40.431851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445844+0000","last_change":"2024-08-18T13:39:39.443848+0000","last_active":"2024-08-18T13:39:39.445844+0000","last_peered":"2024-08-18T13:39:39.445844+0000","last_clean":"2024-08-18T13:39:39.445844+0000","last_became_active":"2024-08-18T13:39:39.443456+0000","last_became_peered":"2024-08-18T13:39:39.443456+0000","last_unstale":"2024-08-18T13:39:39.445844+0000","last_undegraded":"2024-08-18T13:39:39.445844+0000","last_fullsized":"2024-08-18T13:39:39.445844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:27.994932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450757+0000","last_change":"2024-08-18T13:39:35.421612+0000","last_active":"2024-08-18T13:39:39.450757+0000","last_peered":"2024-08-18T13:39:39.450757+0000","last_clean":"2024-08-18T13:39:39.450757+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:39:39.450757+0000","last_undegraded":"2024-08-18T13:39:39.450757+0000","last_fullsized":"2024-08-18T13:39:39.450757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433442+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:39:40.433442+0000","last_peered":"2024-08-18T13:39:40.433442+0000","last_clean":"2024-08-18T13:39:40.433442+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:39:40.433442+0000","last_undegraded":"2024-08-18T13:39:40.433442+0000","last_fullsized":"2024-08-18T13:39:40.433442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454815+0000","last_change":"2024-08-18T13:39:39.453520+0000","last_active":"2024-08-18T13:39:39.454815+0000","last_peered":"2024-08-18T13:39:39.454815+0000","last_clean":"2024-08-18T13:39:39.454815+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:39:39.454815+0000","last_undegraded":"2024-08-18T13:39:39.454815+0000","last_fullsized":"2024-08-18T13:39:39.454815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450976+0000","last_change":"2024-08-18T13:39:35.421465+0000","last_active":"2024-08-18T13:39:39.450976+0000","last_peered":"2024-08-18T13:39:39.450976+0000","last_clean":"2024-08-18T13:39:39.450976+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:39:39.450976+0000","last_undegraded":"2024-08-18T13:39:39.450976+0000","last_fullsized":"2024-08-18T13:39:39.450976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429394+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:39:40.429394+0000","last_peered":"2024-08-18T13:39:40.429394+0000","last_clean":"2024-08-18T13:39:40.429394+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:39:40.429394+0000","last_undegraded":"2024-08-18T13:39:40.429394+0000","last_fullsized":"2024-08-18T13:39:40.429394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.448067+0000","last_change":"2024-08-18T13:39:39.446029+0000","last_active":"2024-08-18T13:39:39.448067+0000","last_peered":"2024-08-18T13:39:39.448067+0000","last_clean":"2024-08-18T13:39:39.448067+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:39:39.448067+0000","last_undegraded":"2024-08-18T13:39:39.448067+0000","last_fullsized":"2024-08-18T13:39:39.448067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450468+0000","last_change":"2024-08-18T13:39:35.417094+0000","last_active":"2024-08-18T13:39:39.450468+0000","last_peered":"2024-08-18T13:39:39.450468+0000","last_clean":"2024-08-18T13:39:39.450468+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:39:39.450468+0000","last_undegraded":"2024-08-18T13:39:39.450468+0000","last_fullsized":"2024-08-18T13:39:39.450468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.436721+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:39:40.436721+0000","last_peered":"2024-08-18T13:39:40.436721+0000","last_clean":"2024-08-18T13:39:40.436721+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:39:40.436721+0000","last_undegraded":"2024-08-18T13:39:40.436721+0000","last_fullsized":"2024-08-18T13:39:40.436721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437633+0000","last_change":"2024-08-18T13:39:39.436992+0000","last_active":"2024-08-18T13:39:39.437633+0000","last_peered":"2024-08-18T13:39:39.437633+0000","last_clean":"2024-08-18T13:39:39.437633+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:39:39.437633+0000","last_undegraded":"2024-08-18T13:39:39.437633+0000","last_fullsized":"2024-08-18T13:39:39.437633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427745+0000","last_change":"2024-08-18T13:39:35.411315+0000","last_active":"2024-08-18T13:39:39.427745+0000","last_peered":"2024-08-18T13:39:39.427745+0000","last_clean":"2024-08-18T13:39:39.427745+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:39:39.427745+0000","last_undegraded":"2024-08-18T13:39:39.427745+0000","last_fullsized":"2024-08-18T13:39:39.427745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428272+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:39:39.428272+0000","last_peered":"2024-08-18T13:39:39.428272+0000","last_clean":"2024-08-18T13:39:39.428272+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:39:39.428272+0000","last_undegraded":"2024-08-18T13:39:39.428272+0000","last_fullsized":"2024-08-18T13:39:39.428272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.458299+0000","last_change":"2024-08-18T13:39:39.456955+0000","last_active":"2024-08-18T13:39:39.458299+0000","last_peered":"2024-08-18T13:39:39.458299+0000","last_clean":"2024-08-18T13:39:39.458299+0000","last_became_active":"2024-08-18T13:39:39.456534+0000","last_became_peered":"2024-08-18T13:39:39.456534+0000","last_unstale":"2024-08-18T13:39:39.458299+0000","last_undegraded":"2024-08-18T13:39:39.458299+0000","last_fullsized":"2024-08-18T13:39:39.458299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:26:42.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450840+0000","last_change":"2024-08-18T13:39:35.417348+0000","last_active":"2024-08-18T13:39:39.450840+0000","last_peered":"2024-08-18T13:39:39.450840+0000","last_clean":"2024-08-18T13:39:39.450840+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:39:39.450840+0000","last_undegraded":"2024-08-18T13:39:39.450840+0000","last_fullsized":"2024-08-18T13:39:39.450840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417725+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:39:39.417725+0000","last_peered":"2024-08-18T13:39:39.417725+0000","last_clean":"2024-08-18T13:39:39.417725+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:39:39.417725+0000","last_undegraded":"2024-08-18T13:39:39.417725+0000","last_fullsized":"2024-08-18T13:39:39.417725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454724+0000","last_change":"2024-08-18T13:39:39.453682+0000","last_active":"2024-08-18T13:39:39.454724+0000","last_peered":"2024-08-18T13:39:39.454724+0000","last_clean":"2024-08-18T13:39:39.454724+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:39:39.454724+0000","last_undegraded":"2024-08-18T13:39:39.454724+0000","last_fullsized":"2024-08-18T13:39:39.454724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427706+0000","last_change":"2024-08-18T13:39:35.407127+0000","last_active":"2024-08-18T13:39:39.427706+0000","last_peered":"2024-08-18T13:39:39.427706+0000","last_clean":"2024-08-18T13:39:39.427706+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:39:39.427706+0000","last_undegraded":"2024-08-18T13:39:39.427706+0000","last_fullsized":"2024-08-18T13:39:39.427706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431523+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:39:40.431523+0000","last_peered":"2024-08-18T13:39:40.431523+0000","last_clean":"2024-08-18T13:39:40.431523+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:39:40.431523+0000","last_undegraded":"2024-08-18T13:39:40.431523+0000","last_fullsized":"2024-08-18T13:39:40.431523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457313+0000","last_change":"2024-08-18T13:39:39.456626+0000","last_active":"2024-08-18T13:39:39.457313+0000","last_peered":"2024-08-18T13:39:39.457313+0000","last_clean":"2024-08-18T13:39:39.457313+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:39:39.457313+0000","last_undegraded":"2024-08-18T13:39:39.457313+0000","last_fullsized":"2024-08-18T13:39:39.457313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450395+0000","last_change":"2024-08-18T13:39:35.417693+0000","last_active":"2024-08-18T13:39:39.450395+0000","last_peered":"2024-08-18T13:39:39.450395+0000","last_clean":"2024-08-18T13:39:39.450395+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:39:39.450395+0000","last_undegraded":"2024-08-18T13:39:39.450395+0000","last_fullsized":"2024-08-18T13:39:39.450395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:48.063001+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:39:48.063001+0000","last_peered":"2024-08-18T13:39:48.063001+0000","last_clean":"2024-08-18T13:39:48.063001+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:39:48.063001+0000","last_undegraded":"2024-08-18T13:39:48.063001+0000","last_fullsized":"2024-08-18T13:39:48.063001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3762,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.440639+0000","last_change":"2024-08-18T13:39:39.438064+0000","last_active":"2024-08-18T13:39:39.440639+0000","last_peered":"2024-08-18T13:39:39.440639+0000","last_clean":"2024-08-18T13:39:39.440639+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:39:39.440639+0000","last_undegraded":"2024-08-18T13:39:39.440639+0000","last_fullsized":"2024-08-18T13:39:39.440639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421839+0000","last_change":"2024-08-18T13:39:35.406398+0000","last_active":"2024-08-18T13:39:39.421839+0000","last_peered":"2024-08-18T13:39:39.421839+0000","last_clean":"2024-08-18T13:39:39.421839+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:39:39.421839+0000","last_undegraded":"2024-08-18T13:39:39.421839+0000","last_fullsized":"2024-08-18T13:39:39.421839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433257+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:39:40.433257+0000","last_peered":"2024-08-18T13:39:40.433257+0000","last_clean":"2024-08-18T13:39:40.433257+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:39:40.433257+0000","last_undegraded":"2024-08-18T13:39:40.433257+0000","last_fullsized":"2024-08-18T13:39:40.433257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444385+0000","last_change":"2024-08-18T13:39:39.439962+0000","last_active":"2024-08-18T13:39:39.444385+0000","last_peered":"2024-08-18T13:39:39.444385+0000","last_clean":"2024-08-18T13:39:39.444385+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:39:39.444385+0000","last_undegraded":"2024-08-18T13:39:39.444385+0000","last_fullsized":"2024-08-18T13:39:39.444385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432863+0000","last_change":"2024-08-18T13:39:35.411067+0000","last_active":"2024-08-18T13:39:39.432863+0000","last_peered":"2024-08-18T13:39:39.432863+0000","last_clean":"2024-08-18T13:39:39.432863+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:39:39.432863+0000","last_undegraded":"2024-08-18T13:39:39.432863+0000","last_fullsized":"2024-08-18T13:39:39.432863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422811+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:39:39.422811+0000","last_peered":"2024-08-18T13:39:39.422811+0000","last_clean":"2024-08-18T13:39:39.422811+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:39:39.422811+0000","last_undegraded":"2024-08-18T13:39:39.422811+0000","last_fullsized":"2024-08-18T13:39:39.422811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447069+0000","last_change":"2024-08-18T13:39:39.443938+0000","last_active":"2024-08-18T13:39:39.447069+0000","last_peered":"2024-08-18T13:39:39.447069+0000","last_clean":"2024-08-18T13:39:39.447069+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:39:39.447069+0000","last_undegraded":"2024-08-18T13:39:39.447069+0000","last_fullsized":"2024-08-18T13:39:39.447069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422769+0000","last_change":"2024-08-18T13:39:35.409517+0000","last_active":"2024-08-18T13:39:39.422769+0000","last_peered":"2024-08-18T13:39:39.422769+0000","last_clean":"2024-08-18T13:39:39.422769+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:39:39.422769+0000","last_undegraded":"2024-08-18T13:39:39.422769+0000","last_fullsized":"2024-08-18T13:39:39.422769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429841+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:39:39.429841+0000","last_peered":"2024-08-18T13:39:39.429841+0000","last_clean":"2024-08-18T13:39:39.429841+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:39:39.429841+0000","last_undegraded":"2024-08-18T13:39:39.429841+0000","last_fullsized":"2024-08-18T13:39:39.429841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450637+0000","last_change":"2024-08-18T13:39:39.449671+0000","last_active":"2024-08-18T13:39:39.450637+0000","last_peered":"2024-08-18T13:39:39.450637+0000","last_clean":"2024-08-18T13:39:39.450637+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:39:39.450637+0000","last_undegraded":"2024-08-18T13:39:39.450637+0000","last_fullsized":"2024-08-18T13:39:39.450637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454824+0000","last_change":"2024-08-18T13:39:39.453907+0000","last_active":"2024-08-18T13:39:39.454824+0000","last_peered":"2024-08-18T13:39:39.454824+0000","last_clean":"2024-08-18T13:39:39.454824+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:39:39.454824+0000","last_undegraded":"2024-08-18T13:39:39.454824+0000","last_fullsized":"2024-08-18T13:39:39.454824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422264+0000","last_change":"2024-08-18T13:39:35.401997+0000","last_active":"2024-08-18T13:39:39.422264+0000","last_peered":"2024-08-18T13:39:39.422264+0000","last_clean":"2024-08-18T13:39:39.422264+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:39:39.422264+0000","last_undegraded":"2024-08-18T13:39:39.422264+0000","last_fullsized":"2024-08-18T13:39:39.422264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418448+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:39:39.418448+0000","last_peered":"2024-08-18T13:39:39.418448+0000","last_clean":"2024-08-18T13:39:39.418448+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:39:39.418448+0000","last_undegraded":"2024-08-18T13:39:39.418448+0000","last_fullsized":"2024-08-18T13:39:39.418448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442977+0000","last_change":"2024-08-18T13:39:39.440811+0000","last_active":"2024-08-18T13:39:39.442977+0000","last_peered":"2024-08-18T13:39:39.442977+0000","last_clean":"2024-08-18T13:39:39.442977+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:39:39.442977+0000","last_undegraded":"2024-08-18T13:39:39.442977+0000","last_fullsized":"2024-08-18T13:39:39.442977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430210+0000","last_change":"2024-08-18T13:39:35.412127+0000","last_active":"2024-08-18T13:39:39.430210+0000","last_peered":"2024-08-18T13:39:39.430210+0000","last_clean":"2024-08-18T13:39:39.430210+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:39:39.430210+0000","last_undegraded":"2024-08-18T13:39:39.430210+0000","last_fullsized":"2024-08-18T13:39:39.430210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423847+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:39:39.423847+0000","last_peered":"2024-08-18T13:39:39.423847+0000","last_clean":"2024-08-18T13:39:39.423847+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:39:39.423847+0000","last_undegraded":"2024-08-18T13:39:39.423847+0000","last_fullsized":"2024-08-18T13:39:39.423847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429678+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:39:40.429678+0000","last_peered":"2024-08-18T13:39:40.429678+0000","last_clean":"2024-08-18T13:39:40.429678+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:39:40.429678+0000","last_undegraded":"2024-08-18T13:39:40.429678+0000","last_fullsized":"2024-08-18T13:39:40.429678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429574+0000","last_change":"2024-08-18T13:39:35.407995+0000","last_active":"2024-08-18T13:39:39.429574+0000","last_peered":"2024-08-18T13:39:39.429574+0000","last_clean":"2024-08-18T13:39:39.429574+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:39:39.429574+0000","last_undegraded":"2024-08-18T13:39:39.429574+0000","last_fullsized":"2024-08-18T13:39:39.429574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.438294+0000","last_change":"2024-08-18T13:39:39.437542+0000","last_active":"2024-08-18T13:39:39.438294+0000","last_peered":"2024-08-18T13:39:39.438294+0000","last_clean":"2024-08-18T13:39:39.438294+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:39:39.438294+0000","last_undegraded":"2024-08-18T13:39:39.438294+0000","last_fullsized":"2024-08-18T13:39:39.438294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434310+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:39:40.434310+0000","last_peered":"2024-08-18T13:39:40.434310+0000","last_clean":"2024-08-18T13:39:40.434310+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:39:40.434310+0000","last_undegraded":"2024-08-18T13:39:40.434310+0000","last_fullsized":"2024-08-18T13:39:40.434310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422760+0000","last_change":"2024-08-18T13:39:35.403759+0000","last_active":"2024-08-18T13:39:39.422760+0000","last_peered":"2024-08-18T13:39:39.422760+0000","last_clean":"2024-08-18T13:39:39.422760+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:39:39.422760+0000","last_undegraded":"2024-08-18T13:39:39.422760+0000","last_fullsized":"2024-08-18T13:39:39.422760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446432+0000","last_change":"2024-08-18T13:39:39.441201+0000","last_active":"2024-08-18T13:39:39.446432+0000","last_peered":"2024-08-18T13:39:39.446432+0000","last_clean":"2024-08-18T13:39:39.446432+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:39:39.446432+0000","last_undegraded":"2024-08-18T13:39:39.446432+0000","last_fullsized":"2024-08-18T13:39:39.446432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427616+0000","last_change":"2024-08-18T13:39:35.413292+0000","last_active":"2024-08-18T13:39:39.427616+0000","last_peered":"2024-08-18T13:39:39.427616+0000","last_clean":"2024-08-18T13:39:39.427616+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:39:39.427616+0000","last_undegraded":"2024-08-18T13:39:39.427616+0000","last_fullsized":"2024-08-18T13:39:39.427616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415812+0000","last_change":"2024-08-18T13:39:35.401196+0000","last_active":"2024-08-18T13:39:39.415812+0000","last_peered":"2024-08-18T13:39:39.415812+0000","last_clean":"2024-08-18T13:39:39.415812+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:39:39.415812+0000","last_undegraded":"2024-08-18T13:39:39.415812+0000","last_fullsized":"2024-08-18T13:39:39.415812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417577+0000","last_change":"2024-08-18T13:39:35.406603+0000","last_active":"2024-08-18T13:39:39.417577+0000","last_peered":"2024-08-18T13:39:39.417577+0000","last_clean":"2024-08-18T13:39:39.417577+0000","last_became_active":"2024-08-18T13:39:35.406331+0000","last_became_peered":"2024-08-18T13:39:35.406331+0000","last_unstale":"2024-08-18T13:39:39.417577+0000","last_undegraded":"2024-08-18T13:39:39.417577+0000","last_fullsized":"2024-08-18T13:39:39.417577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:21:12.232172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429494+0000","last_change":"2024-08-18T13:39:35.404051+0000","last_active":"2024-08-18T13:39:39.429494+0000","last_peered":"2024-08-18T13:39:39.429494+0000","last_clean":"2024-08-18T13:39:39.429494+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:39:39.429494+0000","last_undegraded":"2024-08-18T13:39:39.429494+0000","last_fullsized":"2024-08-18T13:39:39.429494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422959+0000","last_change":"2024-08-18T13:39:35.409288+0000","last_active":"2024-08-18T13:39:39.422959+0000","last_peered":"2024-08-18T13:39:39.422959+0000","last_clean":"2024-08-18T13:39:39.422959+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:39:39.422959+0000","last_undegraded":"2024-08-18T13:39:39.422959+0000","last_fullsized":"2024-08-18T13:39:39.422959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424121+0000","last_change":"2024-08-18T13:39:35.403051+0000","last_active":"2024-08-18T13:39:39.424121+0000","last_peered":"2024-08-18T13:39:39.424121+0000","last_clean":"2024-08-18T13:39:39.424121+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:39:39.424121+0000","last_undegraded":"2024-08-18T13:39:39.424121+0000","last_fullsized":"2024-08-18T13:39:39.424121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430049+0000","last_change":"2024-08-18T13:39:35.412281+0000","last_active":"2024-08-18T13:39:39.430049+0000","last_peered":"2024-08-18T13:39:39.430049+0000","last_clean":"2024-08-18T13:39:39.430049+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:39:39.430049+0000","last_undegraded":"2024-08-18T13:39:39.430049+0000","last_fullsized":"2024-08-18T13:39:39.430049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424466+0000","last_change":"2024-08-18T13:39:35.409220+0000","last_active":"2024-08-18T13:39:39.424466+0000","last_peered":"2024-08-18T13:39:39.424466+0000","last_clean":"2024-08-18T13:39:39.424466+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:39:39.424466+0000","last_undegraded":"2024-08-18T13:39:39.424466+0000","last_fullsized":"2024-08-18T13:39:39.424466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418253+0000","last_change":"2024-08-18T13:39:35.401759+0000","last_active":"2024-08-18T13:39:39.418253+0000","last_peered":"2024-08-18T13:39:39.418253+0000","last_clean":"2024-08-18T13:39:39.418253+0000","last_became_active":"2024-08-18T13:39:35.401476+0000","last_became_peered":"2024-08-18T13:39:35.401476+0000","last_unstale":"2024-08-18T13:39:39.418253+0000","last_undegraded":"2024-08-18T13:39:39.418253+0000","last_fullsized":"2024-08-18T13:39:39.418253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:04:43.997597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423669+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.423669+0000","last_peered":"2024-08-18T13:39:39.423669+0000","last_clean":"2024-08-18T13:39:39.423669+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:39:39.423669+0000","last_undegraded":"2024-08-18T13:39:39.423669+0000","last_fullsized":"2024-08-18T13:39:39.423669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417410+0000","last_change":"2024-08-18T13:39:35.407218+0000","last_active":"2024-08-18T13:39:39.417410+0000","last_peered":"2024-08-18T13:39:39.417410+0000","last_clean":"2024-08-18T13:39:39.417410+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:39:39.417410+0000","last_undegraded":"2024-08-18T13:39:39.417410+0000","last_fullsized":"2024-08-18T13:39:39.417410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429387+0000","last_change":"2024-08-18T13:39:35.407339+0000","last_active":"2024-08-18T13:39:39.429387+0000","last_peered":"2024-08-18T13:39:39.429387+0000","last_clean":"2024-08-18T13:39:39.429387+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:39:39.429387+0000","last_undegraded":"2024-08-18T13:39:39.429387+0000","last_fullsized":"2024-08-18T13:39:39.429387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429062+0000","last_change":"2024-08-18T13:39:35.408647+0000","last_active":"2024-08-18T13:39:39.429062+0000","last_peered":"2024-08-18T13:39:39.429062+0000","last_clean":"2024-08-18T13:39:39.429062+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:39:39.429062+0000","last_undegraded":"2024-08-18T13:39:39.429062+0000","last_fullsized":"2024-08-18T13:39:39.429062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450837+0000","last_change":"2024-08-18T13:39:35.418000+0000","last_active":"2024-08-18T13:39:39.450837+0000","last_peered":"2024-08-18T13:39:39.450837+0000","last_clean":"2024-08-18T13:39:39.450837+0000","last_became_active":"2024-08-18T13:39:35.417661+0000","last_became_peered":"2024-08-18T13:39:35.417661+0000","last_unstale":"2024-08-18T13:39:39.450837+0000","last_undegraded":"2024-08-18T13:39:39.450837+0000","last_fullsized":"2024-08-18T13:39:39.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:53:44.739585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416124+0000","last_change":"2024-08-18T13:39:35.405425+0000","last_active":"2024-08-18T13:39:39.416124+0000","last_peered":"2024-08-18T13:39:39.416124+0000","last_clean":"2024-08-18T13:39:39.416124+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:39:39.416124+0000","last_undegraded":"2024-08-18T13:39:39.416124+0000","last_fullsized":"2024-08-18T13:39:39.416124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423886+0000","last_change":"2024-08-18T13:39:35.400889+0000","last_active":"2024-08-18T13:39:39.423886+0000","last_peered":"2024-08-18T13:39:39.423886+0000","last_clean":"2024-08-18T13:39:39.423886+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:39:39.423886+0000","last_undegraded":"2024-08-18T13:39:39.423886+0000","last_fullsized":"2024-08-18T13:39:39.423886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418057+0000","last_change":"2024-08-18T13:39:35.407634+0000","last_active":"2024-08-18T13:39:39.418057+0000","last_peered":"2024-08-18T13:39:39.418057+0000","last_clean":"2024-08-18T13:39:39.418057+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:39:39.418057+0000","last_undegraded":"2024-08-18T13:39:39.418057+0000","last_fullsized":"2024-08-18T13:39:39.418057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422473+0000","last_change":"2024-08-18T13:39:35.397509+0000","last_active":"2024-08-18T13:39:39.422473+0000","last_peered":"2024-08-18T13:39:39.422473+0000","last_clean":"2024-08-18T13:39:39.422473+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:39:39.422473+0000","last_undegraded":"2024-08-18T13:39:39.422473+0000","last_fullsized":"2024-08-18T13:39:39.422473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425330+0000","last_change":"2024-08-18T13:39:35.404533+0000","last_active":"2024-08-18T13:39:39.425330+0000","last_peered":"2024-08-18T13:39:39.425330+0000","last_clean":"2024-08-18T13:39:39.425330+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:39:39.425330+0000","last_undegraded":"2024-08-18T13:39:39.425330+0000","last_fullsized":"2024-08-18T13:39:39.425330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425567+0000","last_change":"2024-08-18T13:39:35.404839+0000","last_active":"2024-08-18T13:39:39.425567+0000","last_peered":"2024-08-18T13:39:39.425567+0000","last_clean":"2024-08-18T13:39:39.425567+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:39:39.425567+0000","last_undegraded":"2024-08-18T13:39:39.425567+0000","last_fullsized":"2024-08-18T13:39:39.425567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421633+0000","last_change":"2024-08-18T13:39:35.405867+0000","last_active":"2024-08-18T13:39:39.421633+0000","last_peered":"2024-08-18T13:39:39.421633+0000","last_clean":"2024-08-18T13:39:39.421633+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:39:39.421633+0000","last_undegraded":"2024-08-18T13:39:39.421633+0000","last_fullsized":"2024-08-18T13:39:39.421633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429697+0000","last_change":"2024-08-18T13:39:35.411173+0000","last_active":"2024-08-18T13:39:39.429697+0000","last_peered":"2024-08-18T13:39:39.429697+0000","last_clean":"2024-08-18T13:39:39.429697+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:39:39.429697+0000","last_undegraded":"2024-08-18T13:39:39.429697+0000","last_fullsized":"2024-08-18T13:39:39.429697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429962+0000","last_change":"2024-08-18T13:39:35.418013+0000","last_active":"2024-08-18T13:39:39.429962+0000","last_peered":"2024-08-18T13:39:39.429962+0000","last_clean":"2024-08-18T13:39:39.429962+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:39:39.429962+0000","last_undegraded":"2024-08-18T13:39:39.429962+0000","last_fullsized":"2024-08-18T13:39:39.429962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450803+0000","last_change":"2024-08-18T13:39:35.421633+0000","last_active":"2024-08-18T13:39:39.450803+0000","last_peered":"2024-08-18T13:39:39.450803+0000","last_clean":"2024-08-18T13:39:39.450803+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:39:39.450803+0000","last_undegraded":"2024-08-18T13:39:39.450803+0000","last_fullsized":"2024-08-18T13:39:39.450803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425234+0000","last_change":"2024-08-18T13:39:35.411443+0000","last_active":"2024-08-18T13:39:39.425234+0000","last_peered":"2024-08-18T13:39:39.425234+0000","last_clean":"2024-08-18T13:39:39.425234+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:39:39.425234+0000","last_undegraded":"2024-08-18T13:39:39.425234+0000","last_fullsized":"2024-08-18T13:39:39.425234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450366+0000","last_change":"2024-08-18T13:39:35.415548+0000","last_active":"2024-08-18T13:39:39.450366+0000","last_peered":"2024-08-18T13:39:39.450366+0000","last_clean":"2024-08-18T13:39:39.450366+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:39:39.450366+0000","last_undegraded":"2024-08-18T13:39:39.450366+0000","last_fullsized":"2024-08-18T13:39:39.450366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417246+0000","last_change":"2024-08-18T13:39:35.410795+0000","last_active":"2024-08-18T13:39:39.417246+0000","last_peered":"2024-08-18T13:39:39.417246+0000","last_clean":"2024-08-18T13:39:39.417246+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:39:39.417246+0000","last_undegraded":"2024-08-18T13:39:39.417246+0000","last_fullsized":"2024-08-18T13:39:39.417246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450710+0000","last_change":"2024-08-18T13:39:35.417550+0000","last_active":"2024-08-18T13:39:39.450710+0000","last_peered":"2024-08-18T13:39:39.450710+0000","last_clean":"2024-08-18T13:39:39.450710+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:39:39.450710+0000","last_undegraded":"2024-08-18T13:39:39.450710+0000","last_fullsized":"2024-08-18T13:39:39.450710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422555+0000","last_change":"2024-08-18T13:39:35.410002+0000","last_active":"2024-08-18T13:39:39.422555+0000","last_peered":"2024-08-18T13:39:39.422555+0000","last_clean":"2024-08-18T13:39:39.422555+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:39:39.422555+0000","last_undegraded":"2024-08-18T13:39:39.422555+0000","last_fullsized":"2024-08-18T13:39:39.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427751+0000","last_change":"2024-08-18T13:39:35.406627+0000","last_active":"2024-08-18T13:39:39.427751+0000","last_peered":"2024-08-18T13:39:39.427751+0000","last_clean":"2024-08-18T13:39:39.427751+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:39:39.427751+0000","last_undegraded":"2024-08-18T13:39:39.427751+0000","last_fullsized":"2024-08-18T13:39:39.427751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423078+0000","last_change":"2024-08-18T13:39:35.405750+0000","last_active":"2024-08-18T13:39:39.423078+0000","last_peered":"2024-08-18T13:39:39.423078+0000","last_clean":"2024-08-18T13:39:39.423078+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:39:39.423078+0000","last_undegraded":"2024-08-18T13:39:39.423078+0000","last_fullsized":"2024-08-18T13:39:39.423078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450591+0000","last_change":"2024-08-18T13:39:35.417195+0000","last_active":"2024-08-18T13:39:39.450591+0000","last_peered":"2024-08-18T13:39:39.450591+0000","last_clean":"2024-08-18T13:39:39.450591+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:39:39.450591+0000","last_undegraded":"2024-08-18T13:39:39.450591+0000","last_fullsized":"2024-08-18T13:39:39.450591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4548,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":856064,"data_stored":693001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]}]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":270336,"data_stored":102189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":258048,"data_stored":102063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":102223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":864256,"data_stored":693057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":262144,"data_stored":102153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:39:58.507 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:39:58.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567eb77700 1 Processor -- start 2024-08-18T13:39:58.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567eb77700 1 -- start start 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567eb77700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678153690 0x7f56780673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567eb77700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f567806f2f0 0x7f56780c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567eb77700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567806f770 con 0x7f56780c8200 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567eb77700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567815a640 con 0x7f567806f2f0 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567eb77700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567806ee60 con 0x7f5678153690 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567e376700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f567806f2f0 0x7f56780c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567db75700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678153690 0x7f56780673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567e376700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f567806f2f0 0x7f56780c29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:43752/0 (socket says 172.21.15.118:43752) 2024-08-18T13:39:58.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.606+0000 7f567e376700 1 -- 172.21.15.118:0/3151354257 learned_addr learned my addr 172.21.15.118:0/3151354257 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567e376700 1 -- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678153690 msgr2=0x7f56780673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567e376700 1 --2- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678153690 0x7f56780673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567e376700 1 -- 172.21.15.118:0/3151354257 >> v1:172.21.15.178:6789/0 conn(0x7f56780c8200 legacy=0x7f56780c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567e376700 1 -- 172.21.15.118:0/3151354257 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678153d10 con 0x7f567806f2f0 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567db75700 1 --2- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678153690 0x7f56780673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567e376700 1 --2- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f567806f2f0 0x7f56780c29b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56740099b0 tx=0x7f5674009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7908f5009232d33 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567cb73700 1 -- 172.21.15.118:0/3151354257 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674009ea0 con 0x7f567806f2f0 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567cb73700 1 -- 172.21.15.118:0/3151354257 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567400b860 con 0x7f567806f2f0 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567cb73700 1 -- 172.21.15.118:0/3151354257 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400bc00 con 0x7f567806f2f0 2024-08-18T13:39:58.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f567806f2f0 msgr2=0x7f56780c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 --2- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f567806f2f0 0x7f56780c29b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56740099b0 tx=0x7f5674009250 comp rx=0 tx=0).stop 2024-08-18T13:39:58.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- 172.21.15.118:0/3151354257 shutdown_connections 2024-08-18T13:39:58.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 --2- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f567806f2f0 0x7f56780c29b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 --2- 172.21.15.118:0/3151354257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678153690 0x7f56780673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- 172.21.15.118:0/3151354257 >> 172.21.15.118:0/3151354257 conn(0x7f567806c020 msgr2=0x7f56780c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:58.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- 172.21.15.118:0/3151354257 shutdown_connections 2024-08-18T13:39:58.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- 172.21.15.118:0/3151354257 wait complete. 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 Processor -- start 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- start start 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f567806f2f0 0x7f5678151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56780c8200 0x7f5678169460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678153690 0x7f567815c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56780c0d30 con 0x7f567806f2f0 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5678156700 con 0x7f5678153690 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.610+0000 7f567eb77700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567815d970 con 0x7f56780c8200 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f567806f2f0 0x7f5678151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f567806f2f0 0x7f5678151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44978/0 (socket says 172.21.15.118:44978) 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 -- 172.21.15.118:0/3053587088 learned_addr learned my addr 172.21.15.118:0/3053587088 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:58.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567d374700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56780c8200 0x7f5678169460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567e376700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678153690 0x7f567815c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 -- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56780c8200 msgr2=0x7f5678169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56780c8200 0x7f5678169460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 -- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678153690 msgr2=0x7f567815c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678153690 0x7f567815c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 -- 172.21.15.118:0/3053587088 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670009a20 con 0x7f567806f2f0 2024-08-18T13:39:58.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567db75700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f567806f2f0 0x7f5678151af0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f567000c0f0 tx=0x7f567000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f566dffb700 1 -- 172.21.15.118:0/3053587088 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000cc30 con 0x7f567806f2f0 2024-08-18T13:39:58.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f566dffb700 1 -- 172.21.15.118:0/3053587088 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567000f040 con 0x7f567806f2f0 2024-08-18T13:39:58.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f566dffb700 1 -- 172.21.15.118:0/3053587088 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670014480 con 0x7f567806f2f0 2024-08-18T13:39:58.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56740094f0 con 0x7f567806f2f0 2024-08-18T13:39:58.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5678068450 con 0x7f567806f2f0 2024-08-18T13:39:58.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.614+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56780713a0 con 0x7f567806f2f0 2024-08-18T13:39:58.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.618+0000 7f566dffb700 1 -- 172.21.15.118:0/3053587088 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f56700147f0 con 0x7f567806f2f0 2024-08-18T13:39:58.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.618+0000 7f566dffb700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f565c060890 0x7f565c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.618+0000 7f566dffb700 1 -- 172.21.15.118:0/3053587088 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f567007e040 con 0x7f567806f2f0 2024-08-18T13:39:58.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.618+0000 7f567d374700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f565c060890 0x7f565c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.618+0000 7f567d374700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f565c060890 0x7f565c062d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5678166ff0 tx=0x7f5668007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.626+0000 7f566dffb700 1 -- 172.21.15.118:0/3053587088 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f567004e220 con 0x7f567806f2f0 2024-08-18T13:39:58.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.786+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5678156840 con 0x7f565c060890 2024-08-18T13:39:58.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.802+0000 7f566dffb700 1 -- 172.21.15.118:0/3053587088 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345217 (secure 0 0 0) 0x7f5668014450 con 0x7f565c060890 2024-08-18T13:39:58.813 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:58.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f565c060890 msgr2=0x7f565c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f565c060890 0x7f565c062d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5678166ff0 tx=0x7f5668007ec0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f567806f2f0 msgr2=0x7f5678151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f567806f2f0 0x7f5678151af0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f567000c0f0 tx=0x7f567000b040 comp rx=0 tx=0).stop 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 shutdown_connections 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f565c060890 0x7f565c062d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5678153690 0x7f567815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56780c8200 0x7f5678169460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 --2- 172.21.15.118:0/3053587088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f567806f2f0 0x7f5678151af0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 >> 172.21.15.118:0/3053587088 conn(0x7f567806c020 msgr2=0x7f567806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 shutdown_connections 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.810+0000 7f567eb77700 1 -- 172.21.15.118:0/3053587088 wait complete. 2024-08-18T13:39:58.817 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:39:58.829 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-08-18T13:39:57.203240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594916,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415368,"kb_used_data":3240,"kb_used_omap":13,"kb_used_meta":2412018,"kb_avail":752559352,"statfs":{"total":773094113280,"available":770620776448,"internally_reserved":0,"allocated":3317760,"data_stored":2006336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2469907136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2210,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004938"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433629+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:39:40.433629+0000","last_peered":"2024-08-18T13:39:40.433629+0000","last_clean":"2024-08-18T13:39:40.433629+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:39:40.433629+0000","last_undegraded":"2024-08-18T13:39:40.433629+0000","last_fullsized":"2024-08-18T13:39:40.433629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422872+0000","last_change":"2024-08-18T13:39:35.405966+0000","last_active":"2024-08-18T13:39:39.422872+0000","last_peered":"2024-08-18T13:39:39.422872+0000","last_clean":"2024-08-18T13:39:39.422872+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:39:39.422872+0000","last_undegraded":"2024-08-18T13:39:39.422872+0000","last_fullsized":"2024-08-18T13:39:39.422872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454715+0000","last_change":"2024-08-18T13:39:39.453674+0000","last_active":"2024-08-18T13:39:39.454715+0000","last_peered":"2024-08-18T13:39:39.454715+0000","last_clean":"2024-08-18T13:39:39.454715+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:39:39.454715+0000","last_undegraded":"2024-08-18T13:39:39.454715+0000","last_fullsized":"2024-08-18T13:39:39.454715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434410+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:39:40.434410+0000","last_peered":"2024-08-18T13:39:40.434410+0000","last_clean":"2024-08-18T13:39:40.434410+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:39:40.434410+0000","last_undegraded":"2024-08-18T13:39:40.434410+0000","last_fullsized":"2024-08-18T13:39:40.434410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432862+0000","last_change":"2024-08-18T13:39:35.408917+0000","last_active":"2024-08-18T13:39:39.432862+0000","last_peered":"2024-08-18T13:39:39.432862+0000","last_clean":"2024-08-18T13:39:39.432862+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:39:39.432862+0000","last_undegraded":"2024-08-18T13:39:39.432862+0000","last_fullsized":"2024-08-18T13:39:39.432862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447766+0000","last_change":"2024-08-18T13:39:39.445388+0000","last_active":"2024-08-18T13:39:39.447766+0000","last_peered":"2024-08-18T13:39:39.447766+0000","last_clean":"2024-08-18T13:39:39.447766+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:39:39.447766+0000","last_undegraded":"2024-08-18T13:39:39.447766+0000","last_fullsized":"2024-08-18T13:39:39.447766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430049+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:39:40.430049+0000","last_peered":"2024-08-18T13:39:40.430049+0000","last_clean":"2024-08-18T13:39:40.430049+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:39:40.430049+0000","last_undegraded":"2024-08-18T13:39:40.430049+0000","last_fullsized":"2024-08-18T13:39:40.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422649+0000","last_change":"2024-08-18T13:39:35.399188+0000","last_active":"2024-08-18T13:39:39.422649+0000","last_peered":"2024-08-18T13:39:39.422649+0000","last_clean":"2024-08-18T13:39:39.422649+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:39:39.422649+0000","last_undegraded":"2024-08-18T13:39:39.422649+0000","last_fullsized":"2024-08-18T13:39:39.422649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447285+0000","last_change":"2024-08-18T13:39:39.440922+0000","last_active":"2024-08-18T13:39:39.447285+0000","last_peered":"2024-08-18T13:39:39.447285+0000","last_clean":"2024-08-18T13:39:39.447285+0000","last_became_active":"2024-08-18T13:39:39.440426+0000","last_became_peered":"2024-08-18T13:39:39.440426+0000","last_unstale":"2024-08-18T13:39:39.447285+0000","last_undegraded":"2024-08-18T13:39:39.447285+0000","last_fullsized":"2024-08-18T13:39:39.447285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:23:00.376805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.428764+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:39:40.428764+0000","last_peered":"2024-08-18T13:39:40.428764+0000","last_clean":"2024-08-18T13:39:40.428764+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:39:40.428764+0000","last_undegraded":"2024-08-18T13:39:40.428764+0000","last_fullsized":"2024-08-18T13:39:40.428764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423184+0000","last_change":"2024-08-18T13:39:35.400038+0000","last_active":"2024-08-18T13:39:39.423184+0000","last_peered":"2024-08-18T13:39:39.423184+0000","last_clean":"2024-08-18T13:39:39.423184+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:39:39.423184+0000","last_undegraded":"2024-08-18T13:39:39.423184+0000","last_fullsized":"2024-08-18T13:39:39.423184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437377+0000","last_change":"2024-08-18T13:39:39.436221+0000","last_active":"2024-08-18T13:39:39.437377+0000","last_peered":"2024-08-18T13:39:39.437377+0000","last_clean":"2024-08-18T13:39:39.437377+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:39:39.437377+0000","last_undegraded":"2024-08-18T13:39:39.437377+0000","last_fullsized":"2024-08-18T13:39:39.437377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428676+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.428676+0000","last_peered":"2024-08-18T13:39:39.428676+0000","last_clean":"2024-08-18T13:39:39.428676+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:39:39.428676+0000","last_undegraded":"2024-08-18T13:39:39.428676+0000","last_fullsized":"2024-08-18T13:39:39.428676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430090+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:39:40.430090+0000","last_peered":"2024-08-18T13:39:40.430090+0000","last_clean":"2024-08-18T13:39:40.430090+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:39:40.430090+0000","last_undegraded":"2024-08-18T13:39:40.430090+0000","last_fullsized":"2024-08-18T13:39:40.430090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.443080+0000","last_change":"2024-08-18T13:39:39.439633+0000","last_active":"2024-08-18T13:39:39.443080+0000","last_peered":"2024-08-18T13:39:39.443080+0000","last_clean":"2024-08-18T13:39:39.443080+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:39:39.443080+0000","last_undegraded":"2024-08-18T13:39:39.443080+0000","last_fullsized":"2024-08-18T13:39:39.443080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423607+0000","last_change":"2024-08-18T13:39:35.407947+0000","last_active":"2024-08-18T13:39:39.423607+0000","last_peered":"2024-08-18T13:39:39.423607+0000","last_clean":"2024-08-18T13:39:39.423607+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:39:39.423607+0000","last_undegraded":"2024-08-18T13:39:39.423607+0000","last_fullsized":"2024-08-18T13:39:39.423607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429156+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:39:39.429156+0000","last_peered":"2024-08-18T13:39:39.429156+0000","last_clean":"2024-08-18T13:39:39.429156+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:39:39.429156+0000","last_undegraded":"2024-08-18T13:39:39.429156+0000","last_fullsized":"2024-08-18T13:39:39.429156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442892+0000","last_change":"2024-08-18T13:39:39.439913+0000","last_active":"2024-08-18T13:39:39.442892+0000","last_peered":"2024-08-18T13:39:39.442892+0000","last_clean":"2024-08-18T13:39:39.442892+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:39:39.442892+0000","last_undegraded":"2024-08-18T13:39:39.442892+0000","last_fullsized":"2024-08-18T13:39:39.442892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430332+0000","last_change":"2024-08-18T13:39:35.403254+0000","last_active":"2024-08-18T13:39:39.430332+0000","last_peered":"2024-08-18T13:39:39.430332+0000","last_clean":"2024-08-18T13:39:39.430332+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:39:39.430332+0000","last_undegraded":"2024-08-18T13:39:39.430332+0000","last_fullsized":"2024-08-18T13:39:39.430332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422689+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:39:39.422689+0000","last_peered":"2024-08-18T13:39:39.422689+0000","last_clean":"2024-08-18T13:39:39.422689+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:39:39.422689+0000","last_undegraded":"2024-08-18T13:39:39.422689+0000","last_fullsized":"2024-08-18T13:39:39.422689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457822+0000","last_change":"2024-08-18T13:39:39.457250+0000","last_active":"2024-08-18T13:39:39.457822+0000","last_peered":"2024-08-18T13:39:39.457822+0000","last_clean":"2024-08-18T13:39:39.457822+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:39:39.457822+0000","last_undegraded":"2024-08-18T13:39:39.457822+0000","last_fullsized":"2024-08-18T13:39:39.457822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425047+0000","last_change":"2024-08-18T13:39:35.401612+0000","last_active":"2024-08-18T13:39:39.425047+0000","last_peered":"2024-08-18T13:39:39.425047+0000","last_clean":"2024-08-18T13:39:39.425047+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:39:39.425047+0000","last_undegraded":"2024-08-18T13:39:39.425047+0000","last_fullsized":"2024-08-18T13:39:39.425047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450234+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:39:39.450234+0000","last_peered":"2024-08-18T13:39:39.450234+0000","last_clean":"2024-08-18T13:39:39.450234+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:39:39.450234+0000","last_undegraded":"2024-08-18T13:39:39.450234+0000","last_fullsized":"2024-08-18T13:39:39.450234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447688+0000","last_change":"2024-08-18T13:39:39.443503+0000","last_active":"2024-08-18T13:39:39.447688+0000","last_peered":"2024-08-18T13:39:39.447688+0000","last_clean":"2024-08-18T13:39:39.447688+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:39:39.447688+0000","last_undegraded":"2024-08-18T13:39:39.447688+0000","last_fullsized":"2024-08-18T13:39:39.447688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432976+0000","last_change":"2024-08-18T13:39:35.407907+0000","last_active":"2024-08-18T13:39:39.432976+0000","last_peered":"2024-08-18T13:39:39.432976+0000","last_clean":"2024-08-18T13:39:39.432976+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:39:39.432976+0000","last_undegraded":"2024-08-18T13:39:39.432976+0000","last_fullsized":"2024-08-18T13:39:39.432976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429181+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:39:40.429181+0000","last_peered":"2024-08-18T13:39:40.429181+0000","last_clean":"2024-08-18T13:39:40.429181+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:39:40.429181+0000","last_undegraded":"2024-08-18T13:39:40.429181+0000","last_fullsized":"2024-08-18T13:39:40.429181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454767+0000","last_change":"2024-08-18T13:39:39.453138+0000","last_active":"2024-08-18T13:39:39.454767+0000","last_peered":"2024-08-18T13:39:39.454767+0000","last_clean":"2024-08-18T13:39:39.454767+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:39:39.454767+0000","last_undegraded":"2024-08-18T13:39:39.454767+0000","last_fullsized":"2024-08-18T13:39:39.454767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424552+0000","last_change":"2024-08-18T13:39:35.404453+0000","last_active":"2024-08-18T13:39:39.424552+0000","last_peered":"2024-08-18T13:39:39.424552+0000","last_clean":"2024-08-18T13:39:39.424552+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:39:39.424552+0000","last_undegraded":"2024-08-18T13:39:39.424552+0000","last_fullsized":"2024-08-18T13:39:39.424552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:41.051281+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:39:41.051281+0000","last_peered":"2024-08-18T13:39:41.051281+0000","last_clean":"2024-08-18T13:39:41.051281+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:39:41.051281+0000","last_undegraded":"2024-08-18T13:39:41.051281+0000","last_fullsized":"2024-08-18T13:39:41.051281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450645+0000","last_change":"2024-08-18T13:39:39.449851+0000","last_active":"2024-08-18T13:39:39.450645+0000","last_peered":"2024-08-18T13:39:39.450645+0000","last_clean":"2024-08-18T13:39:39.450645+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:39:39.450645+0000","last_undegraded":"2024-08-18T13:39:39.450645+0000","last_fullsized":"2024-08-18T13:39:39.450645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424459+0000","last_change":"2024-08-18T13:39:35.400604+0000","last_active":"2024-08-18T13:39:39.424459+0000","last_peered":"2024-08-18T13:39:39.424459+0000","last_clean":"2024-08-18T13:39:39.424459+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:39:39.424459+0000","last_undegraded":"2024-08-18T13:39:39.424459+0000","last_fullsized":"2024-08-18T13:39:39.424459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450871+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:39:39.450871+0000","last_peered":"2024-08-18T13:39:39.450871+0000","last_clean":"2024-08-18T13:39:39.450871+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:39:39.450871+0000","last_undegraded":"2024-08-18T13:39:39.450871+0000","last_fullsized":"2024-08-18T13:39:39.450871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454096+0000","last_change":"2024-08-18T13:39:39.453599+0000","last_active":"2024-08-18T13:39:39.454096+0000","last_peered":"2024-08-18T13:39:39.454096+0000","last_clean":"2024-08-18T13:39:39.454096+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:39:39.454096+0000","last_undegraded":"2024-08-18T13:39:39.454096+0000","last_fullsized":"2024-08-18T13:39:39.454096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423160+0000","last_change":"2024-08-18T13:39:35.410167+0000","last_active":"2024-08-18T13:39:39.423160+0000","last_peered":"2024-08-18T13:39:39.423160+0000","last_clean":"2024-08-18T13:39:39.423160+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:39:39.423160+0000","last_undegraded":"2024-08-18T13:39:39.423160+0000","last_fullsized":"2024-08-18T13:39:39.423160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424551+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:39:39.424551+0000","last_peered":"2024-08-18T13:39:39.424551+0000","last_clean":"2024-08-18T13:39:39.424551+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:39:39.424551+0000","last_undegraded":"2024-08-18T13:39:39.424551+0000","last_fullsized":"2024-08-18T13:39:39.424551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444517+0000","last_change":"2024-08-18T13:39:39.439948+0000","last_active":"2024-08-18T13:39:39.444517+0000","last_peered":"2024-08-18T13:39:39.444517+0000","last_clean":"2024-08-18T13:39:39.444517+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:39:39.444517+0000","last_undegraded":"2024-08-18T13:39:39.444517+0000","last_fullsized":"2024-08-18T13:39:39.444517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422894+0000","last_change":"2024-08-18T13:39:35.407829+0000","last_active":"2024-08-18T13:39:39.422894+0000","last_peered":"2024-08-18T13:39:39.422894+0000","last_clean":"2024-08-18T13:39:39.422894+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:39:39.422894+0000","last_undegraded":"2024-08-18T13:39:39.422894+0000","last_fullsized":"2024-08-18T13:39:39.422894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417877+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:39:39.417877+0000","last_peered":"2024-08-18T13:39:39.417877+0000","last_clean":"2024-08-18T13:39:39.417877+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:39:39.417877+0000","last_undegraded":"2024-08-18T13:39:39.417877+0000","last_fullsized":"2024-08-18T13:39:39.417877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445479+0000","last_change":"2024-08-18T13:39:39.442372+0000","last_active":"2024-08-18T13:39:39.445479+0000","last_peered":"2024-08-18T13:39:39.445479+0000","last_clean":"2024-08-18T13:39:39.445479+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:39:39.445479+0000","last_undegraded":"2024-08-18T13:39:39.445479+0000","last_fullsized":"2024-08-18T13:39:39.445479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421862+0000","last_change":"2024-08-18T13:39:35.400384+0000","last_active":"2024-08-18T13:39:39.421862+0000","last_peered":"2024-08-18T13:39:39.421862+0000","last_clean":"2024-08-18T13:39:39.421862+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:39:39.421862+0000","last_undegraded":"2024-08-18T13:39:39.421862+0000","last_fullsized":"2024-08-18T13:39:39.421862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415832+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:39:39.415832+0000","last_peered":"2024-08-18T13:39:39.415832+0000","last_clean":"2024-08-18T13:39:39.415832+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:39:39.415832+0000","last_undegraded":"2024-08-18T13:39:39.415832+0000","last_fullsized":"2024-08-18T13:39:39.415832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446859+0000","last_change":"2024-08-18T13:39:39.446401+0000","last_active":"2024-08-18T13:39:39.446859+0000","last_peered":"2024-08-18T13:39:39.446859+0000","last_clean":"2024-08-18T13:39:39.446859+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:39:39.446859+0000","last_undegraded":"2024-08-18T13:39:39.446859+0000","last_fullsized":"2024-08-18T13:39:39.446859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424307+0000","last_change":"2024-08-18T13:39:35.410127+0000","last_active":"2024-08-18T13:39:39.424307+0000","last_peered":"2024-08-18T13:39:39.424307+0000","last_clean":"2024-08-18T13:39:39.424307+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:39:39.424307+0000","last_undegraded":"2024-08-18T13:39:39.424307+0000","last_fullsized":"2024-08-18T13:39:39.424307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450338+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:39:39.450338+0000","last_peered":"2024-08-18T13:39:39.450338+0000","last_clean":"2024-08-18T13:39:39.450338+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:39:39.450338+0000","last_undegraded":"2024-08-18T13:39:39.450338+0000","last_fullsized":"2024-08-18T13:39:39.450338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454861+0000","last_change":"2024-08-18T13:39:39.453901+0000","last_active":"2024-08-18T13:39:39.454861+0000","last_peered":"2024-08-18T13:39:39.454861+0000","last_clean":"2024-08-18T13:39:39.454861+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:39:39.454861+0000","last_undegraded":"2024-08-18T13:39:39.454861+0000","last_fullsized":"2024-08-18T13:39:39.454861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430338+0000","last_change":"2024-08-18T13:39:35.412156+0000","last_active":"2024-08-18T13:39:39.430338+0000","last_peered":"2024-08-18T13:39:39.430338+0000","last_clean":"2024-08-18T13:39:39.430338+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:39:39.430338+0000","last_undegraded":"2024-08-18T13:39:39.430338+0000","last_fullsized":"2024-08-18T13:39:39.430338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417339+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:39:39.417339+0000","last_peered":"2024-08-18T13:39:39.417339+0000","last_clean":"2024-08-18T13:39:39.417339+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:39:39.417339+0000","last_undegraded":"2024-08-18T13:39:39.417339+0000","last_fullsized":"2024-08-18T13:39:39.417339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450727+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:39:39.450727+0000","last_peered":"2024-08-18T13:39:39.450727+0000","last_clean":"2024-08-18T13:39:39.450727+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:39:39.450727+0000","last_undegraded":"2024-08-18T13:39:39.450727+0000","last_fullsized":"2024-08-18T13:39:39.450727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454332+0000","last_change":"2024-08-18T13:39:39.453360+0000","last_active":"2024-08-18T13:39:39.454332+0000","last_peered":"2024-08-18T13:39:39.454332+0000","last_clean":"2024-08-18T13:39:39.454332+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:39:39.454332+0000","last_undegraded":"2024-08-18T13:39:39.454332+0000","last_fullsized":"2024-08-18T13:39:39.454332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450556+0000","last_change":"2024-08-18T13:39:35.421765+0000","last_active":"2024-08-18T13:39:39.450556+0000","last_peered":"2024-08-18T13:39:39.450556+0000","last_clean":"2024-08-18T13:39:39.450556+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:39:39.450556+0000","last_undegraded":"2024-08-18T13:39:39.450556+0000","last_fullsized":"2024-08-18T13:39:39.450556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416404+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:39:39.416404+0000","last_peered":"2024-08-18T13:39:39.416404+0000","last_clean":"2024-08-18T13:39:39.416404+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:39:39.416404+0000","last_undegraded":"2024-08-18T13:39:39.416404+0000","last_fullsized":"2024-08-18T13:39:39.416404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450222+0000","last_change":"2024-08-18T13:39:39.449653+0000","last_active":"2024-08-18T13:39:39.450222+0000","last_peered":"2024-08-18T13:39:39.450222+0000","last_clean":"2024-08-18T13:39:39.450222+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:39:39.450222+0000","last_undegraded":"2024-08-18T13:39:39.450222+0000","last_fullsized":"2024-08-18T13:39:39.450222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422710+0000","last_change":"2024-08-18T13:39:35.407439+0000","last_active":"2024-08-18T13:39:39.422710+0000","last_peered":"2024-08-18T13:39:39.422710+0000","last_clean":"2024-08-18T13:39:39.422710+0000","last_became_active":"2024-08-18T13:39:35.407128+0000","last_became_peered":"2024-08-18T13:39:35.407128+0000","last_unstale":"2024-08-18T13:39:39.422710+0000","last_undegraded":"2024-08-18T13:39:39.422710+0000","last_fullsized":"2024-08-18T13:39:39.422710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431851+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:39:40.431851+0000","last_peered":"2024-08-18T13:39:40.431851+0000","last_clean":"2024-08-18T13:39:40.431851+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:39:40.431851+0000","last_undegraded":"2024-08-18T13:39:40.431851+0000","last_fullsized":"2024-08-18T13:39:40.431851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445844+0000","last_change":"2024-08-18T13:39:39.443848+0000","last_active":"2024-08-18T13:39:39.445844+0000","last_peered":"2024-08-18T13:39:39.445844+0000","last_clean":"2024-08-18T13:39:39.445844+0000","last_became_active":"2024-08-18T13:39:39.443456+0000","last_became_peered":"2024-08-18T13:39:39.443456+0000","last_unstale":"2024-08-18T13:39:39.445844+0000","last_undegraded":"2024-08-18T13:39:39.445844+0000","last_fullsized":"2024-08-18T13:39:39.445844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:27.994932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450757+0000","last_change":"2024-08-18T13:39:35.421612+0000","last_active":"2024-08-18T13:39:39.450757+0000","last_peered":"2024-08-18T13:39:39.450757+0000","last_clean":"2024-08-18T13:39:39.450757+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:39:39.450757+0000","last_undegraded":"2024-08-18T13:39:39.450757+0000","last_fullsized":"2024-08-18T13:39:39.450757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433442+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:39:40.433442+0000","last_peered":"2024-08-18T13:39:40.433442+0000","last_clean":"2024-08-18T13:39:40.433442+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:39:40.433442+0000","last_undegraded":"2024-08-18T13:39:40.433442+0000","last_fullsized":"2024-08-18T13:39:40.433442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454815+0000","last_change":"2024-08-18T13:39:39.453520+0000","last_active":"2024-08-18T13:39:39.454815+0000","last_peered":"2024-08-18T13:39:39.454815+0000","last_clean":"2024-08-18T13:39:39.454815+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:39:39.454815+0000","last_undegraded":"2024-08-18T13:39:39.454815+0000","last_fullsized":"2024-08-18T13:39:39.454815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450976+0000","last_change":"2024-08-18T13:39:35.421465+0000","last_active":"2024-08-18T13:39:39.450976+0000","last_peered":"2024-08-18T13:39:39.450976+0000","last_clean":"2024-08-18T13:39:39.450976+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:39:39.450976+0000","last_undegraded":"2024-08-18T13:39:39.450976+0000","last_fullsized":"2024-08-18T13:39:39.450976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429394+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:39:40.429394+0000","last_peered":"2024-08-18T13:39:40.429394+0000","last_clean":"2024-08-18T13:39:40.429394+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:39:40.429394+0000","last_undegraded":"2024-08-18T13:39:40.429394+0000","last_fullsized":"2024-08-18T13:39:40.429394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.448067+0000","last_change":"2024-08-18T13:39:39.446029+0000","last_active":"2024-08-18T13:39:39.448067+0000","last_peered":"2024-08-18T13:39:39.448067+0000","last_clean":"2024-08-18T13:39:39.448067+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:39:39.448067+0000","last_undegraded":"2024-08-18T13:39:39.448067+0000","last_fullsized":"2024-08-18T13:39:39.448067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450468+0000","last_change":"2024-08-18T13:39:35.417094+0000","last_active":"2024-08-18T13:39:39.450468+0000","last_peered":"2024-08-18T13:39:39.450468+0000","last_clean":"2024-08-18T13:39:39.450468+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:39:39.450468+0000","last_undegraded":"2024-08-18T13:39:39.450468+0000","last_fullsized":"2024-08-18T13:39:39.450468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.436721+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:39:40.436721+0000","last_peered":"2024-08-18T13:39:40.436721+0000","last_clean":"2024-08-18T13:39:40.436721+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:39:40.436721+0000","last_undegraded":"2024-08-18T13:39:40.436721+0000","last_fullsized":"2024-08-18T13:39:40.436721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437633+0000","last_change":"2024-08-18T13:39:39.436992+0000","last_active":"2024-08-18T13:39:39.437633+0000","last_peered":"2024-08-18T13:39:39.437633+0000","last_clean":"2024-08-18T13:39:39.437633+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:39:39.437633+0000","last_undegraded":"2024-08-18T13:39:39.437633+0000","last_fullsized":"2024-08-18T13:39:39.437633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427745+0000","last_change":"2024-08-18T13:39:35.411315+0000","last_active":"2024-08-18T13:39:39.427745+0000","last_peered":"2024-08-18T13:39:39.427745+0000","last_clean":"2024-08-18T13:39:39.427745+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:39:39.427745+0000","last_undegraded":"2024-08-18T13:39:39.427745+0000","last_fullsized":"2024-08-18T13:39:39.427745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428272+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:39:39.428272+0000","last_peered":"2024-08-18T13:39:39.428272+0000","last_clean":"2024-08-18T13:39:39.428272+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:39:39.428272+0000","last_undegraded":"2024-08-18T13:39:39.428272+0000","last_fullsized":"2024-08-18T13:39:39.428272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.458299+0000","last_change":"2024-08-18T13:39:39.456955+0000","last_active":"2024-08-18T13:39:39.458299+0000","last_peered":"2024-08-18T13:39:39.458299+0000","last_clean":"2024-08-18T13:39:39.458299+0000","last_became_active":"2024-08-18T13:39:39.456534+0000","last_became_peered":"2024-08-18T13:39:39.456534+0000","last_unstale":"2024-08-18T13:39:39.458299+0000","last_undegraded":"2024-08-18T13:39:39.458299+0000","last_fullsized":"2024-08-18T13:39:39.458299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:26:42.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450840+0000","last_change":"2024-08-18T13:39:35.417348+0000","last_active":"2024-08-18T13:39:39.450840+0000","last_peered":"2024-08-18T13:39:39.450840+0000","last_clean":"2024-08-18T13:39:39.450840+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:39:39.450840+0000","last_undegraded":"2024-08-18T13:39:39.450840+0000","last_fullsized":"2024-08-18T13:39:39.450840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417725+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:39:39.417725+0000","last_peered":"2024-08-18T13:39:39.417725+0000","last_clean":"2024-08-18T13:39:39.417725+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:39:39.417725+0000","last_undegraded":"2024-08-18T13:39:39.417725+0000","last_fullsized":"2024-08-18T13:39:39.417725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454724+0000","last_change":"2024-08-18T13:39:39.453682+0000","last_active":"2024-08-18T13:39:39.454724+0000","last_peered":"2024-08-18T13:39:39.454724+0000","last_clean":"2024-08-18T13:39:39.454724+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:39:39.454724+0000","last_undegraded":"2024-08-18T13:39:39.454724+0000","last_fullsized":"2024-08-18T13:39:39.454724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427706+0000","last_change":"2024-08-18T13:39:35.407127+0000","last_active":"2024-08-18T13:39:39.427706+0000","last_peered":"2024-08-18T13:39:39.427706+0000","last_clean":"2024-08-18T13:39:39.427706+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:39:39.427706+0000","last_undegraded":"2024-08-18T13:39:39.427706+0000","last_fullsized":"2024-08-18T13:39:39.427706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431523+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:39:40.431523+0000","last_peered":"2024-08-18T13:39:40.431523+0000","last_clean":"2024-08-18T13:39:40.431523+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:39:40.431523+0000","last_undegraded":"2024-08-18T13:39:40.431523+0000","last_fullsized":"2024-08-18T13:39:40.431523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457313+0000","last_change":"2024-08-18T13:39:39.456626+0000","last_active":"2024-08-18T13:39:39.457313+0000","last_peered":"2024-08-18T13:39:39.457313+0000","last_clean":"2024-08-18T13:39:39.457313+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:39:39.457313+0000","last_undegraded":"2024-08-18T13:39:39.457313+0000","last_fullsized":"2024-08-18T13:39:39.457313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450395+0000","last_change":"2024-08-18T13:39:35.417693+0000","last_active":"2024-08-18T13:39:39.450395+0000","last_peered":"2024-08-18T13:39:39.450395+0000","last_clean":"2024-08-18T13:39:39.450395+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:39:39.450395+0000","last_undegraded":"2024-08-18T13:39:39.450395+0000","last_fullsized":"2024-08-18T13:39:39.450395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:48.063001+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:39:48.063001+0000","last_peered":"2024-08-18T13:39:48.063001+0000","last_clean":"2024-08-18T13:39:48.063001+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:39:48.063001+0000","last_undegraded":"2024-08-18T13:39:48.063001+0000","last_fullsized":"2024-08-18T13:39:48.063001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3762,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.440639+0000","last_change":"2024-08-18T13:39:39.438064+0000","last_active":"2024-08-18T13:39:39.440639+0000","last_peered":"2024-08-18T13:39:39.440639+0000","last_clean":"2024-08-18T13:39:39.440639+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:39:39.440639+0000","last_undegraded":"2024-08-18T13:39:39.440639+0000","last_fullsized":"2024-08-18T13:39:39.440639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421839+0000","last_change":"2024-08-18T13:39:35.406398+0000","last_active":"2024-08-18T13:39:39.421839+0000","last_peered":"2024-08-18T13:39:39.421839+0000","last_clean":"2024-08-18T13:39:39.421839+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:39:39.421839+0000","last_undegraded":"2024-08-18T13:39:39.421839+0000","last_fullsized":"2024-08-18T13:39:39.421839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433257+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:39:40.433257+0000","last_peered":"2024-08-18T13:39:40.433257+0000","last_clean":"2024-08-18T13:39:40.433257+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:39:40.433257+0000","last_undegraded":"2024-08-18T13:39:40.433257+0000","last_fullsized":"2024-08-18T13:39:40.433257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444385+0000","last_change":"2024-08-18T13:39:39.439962+0000","last_active":"2024-08-18T13:39:39.444385+0000","last_peered":"2024-08-18T13:39:39.444385+0000","last_clean":"2024-08-18T13:39:39.444385+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:39:39.444385+0000","last_undegraded":"2024-08-18T13:39:39.444385+0000","last_fullsized":"2024-08-18T13:39:39.444385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432863+0000","last_change":"2024-08-18T13:39:35.411067+0000","last_active":"2024-08-18T13:39:39.432863+0000","last_peered":"2024-08-18T13:39:39.432863+0000","last_clean":"2024-08-18T13:39:39.432863+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:39:39.432863+0000","last_undegraded":"2024-08-18T13:39:39.432863+0000","last_fullsized":"2024-08-18T13:39:39.432863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422811+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:39:39.422811+0000","last_peered":"2024-08-18T13:39:39.422811+0000","last_clean":"2024-08-18T13:39:39.422811+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:39:39.422811+0000","last_undegraded":"2024-08-18T13:39:39.422811+0000","last_fullsized":"2024-08-18T13:39:39.422811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447069+0000","last_change":"2024-08-18T13:39:39.443938+0000","last_active":"2024-08-18T13:39:39.447069+0000","last_peered":"2024-08-18T13:39:39.447069+0000","last_clean":"2024-08-18T13:39:39.447069+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:39:39.447069+0000","last_undegraded":"2024-08-18T13:39:39.447069+0000","last_fullsized":"2024-08-18T13:39:39.447069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422769+0000","last_change":"2024-08-18T13:39:35.409517+0000","last_active":"2024-08-18T13:39:39.422769+0000","last_peered":"2024-08-18T13:39:39.422769+0000","last_clean":"2024-08-18T13:39:39.422769+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:39:39.422769+0000","last_undegraded":"2024-08-18T13:39:39.422769+0000","last_fullsized":"2024-08-18T13:39:39.422769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429841+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:39:39.429841+0000","last_peered":"2024-08-18T13:39:39.429841+0000","last_clean":"2024-08-18T13:39:39.429841+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:39:39.429841+0000","last_undegraded":"2024-08-18T13:39:39.429841+0000","last_fullsized":"2024-08-18T13:39:39.429841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450637+0000","last_change":"2024-08-18T13:39:39.449671+0000","last_active":"2024-08-18T13:39:39.450637+0000","last_peered":"2024-08-18T13:39:39.450637+0000","last_clean":"2024-08-18T13:39:39.450637+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:39:39.450637+0000","last_undegraded":"2024-08-18T13:39:39.450637+0000","last_fullsized":"2024-08-18T13:39:39.450637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454824+0000","last_change":"2024-08-18T13:39:39.453907+0000","last_active":"2024-08-18T13:39:39.454824+0000","last_peered":"2024-08-18T13:39:39.454824+0000","last_clean":"2024-08-18T13:39:39.454824+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:39:39.454824+0000","last_undegraded":"2024-08-18T13:39:39.454824+0000","last_fullsized":"2024-08-18T13:39:39.454824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422264+0000","last_change":"2024-08-18T13:39:35.401997+0000","last_active":"2024-08-18T13:39:39.422264+0000","last_peered":"2024-08-18T13:39:39.422264+0000","last_clean":"2024-08-18T13:39:39.422264+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:39:39.422264+0000","last_undegraded":"2024-08-18T13:39:39.422264+0000","last_fullsized":"2024-08-18T13:39:39.422264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418448+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:39:39.418448+0000","last_peered":"2024-08-18T13:39:39.418448+0000","last_clean":"2024-08-18T13:39:39.418448+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:39:39.418448+0000","last_undegraded":"2024-08-18T13:39:39.418448+0000","last_fullsized":"2024-08-18T13:39:39.418448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442977+0000","last_change":"2024-08-18T13:39:39.440811+0000","last_active":"2024-08-18T13:39:39.442977+0000","last_peered":"2024-08-18T13:39:39.442977+0000","last_clean":"2024-08-18T13:39:39.442977+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:39:39.442977+0000","last_undegraded":"2024-08-18T13:39:39.442977+0000","last_fullsized":"2024-08-18T13:39:39.442977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430210+0000","last_change":"2024-08-18T13:39:35.412127+0000","last_active":"2024-08-18T13:39:39.430210+0000","last_peered":"2024-08-18T13:39:39.430210+0000","last_clean":"2024-08-18T13:39:39.430210+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:39:39.430210+0000","last_undegraded":"2024-08-18T13:39:39.430210+0000","last_fullsized":"2024-08-18T13:39:39.430210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423847+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:39:39.423847+0000","last_peered":"2024-08-18T13:39:39.423847+0000","last_clean":"2024-08-18T13:39:39.423847+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:39:39.423847+0000","last_undegraded":"2024-08-18T13:39:39.423847+0000","last_fullsized":"2024-08-18T13:39:39.423847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429678+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:39:40.429678+0000","last_peered":"2024-08-18T13:39:40.429678+0000","last_clean":"2024-08-18T13:39:40.429678+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:39:40.429678+0000","last_undegraded":"2024-08-18T13:39:40.429678+0000","last_fullsized":"2024-08-18T13:39:40.429678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429574+0000","last_change":"2024-08-18T13:39:35.407995+0000","last_active":"2024-08-18T13:39:39.429574+0000","last_peered":"2024-08-18T13:39:39.429574+0000","last_clean":"2024-08-18T13:39:39.429574+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:39:39.429574+0000","last_undegraded":"2024-08-18T13:39:39.429574+0000","last_fullsized":"2024-08-18T13:39:39.429574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.438294+0000","last_change":"2024-08-18T13:39:39.437542+0000","last_active":"2024-08-18T13:39:39.438294+0000","last_peered":"2024-08-18T13:39:39.438294+0000","last_clean":"2024-08-18T13:39:39.438294+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:39:39.438294+0000","last_undegraded":"2024-08-18T13:39:39.438294+0000","last_fullsized":"2024-08-18T13:39:39.438294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434310+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:39:40.434310+0000","last_peered":"2024-08-18T13:39:40.434310+0000","last_clean":"2024-08-18T13:39:40.434310+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:39:40.434310+0000","last_undegraded":"2024-08-18T13:39:40.434310+0000","last_fullsized":"2024-08-18T13:39:40.434310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422760+0000","last_change":"2024-08-18T13:39:35.403759+0000","last_active":"2024-08-18T13:39:39.422760+0000","last_peered":"2024-08-18T13:39:39.422760+0000","last_clean":"2024-08-18T13:39:39.422760+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:39:39.422760+0000","last_undegraded":"2024-08-18T13:39:39.422760+0000","last_fullsized":"2024-08-18T13:39:39.422760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446432+0000","last_change":"2024-08-18T13:39:39.441201+0000","last_active":"2024-08-18T13:39:39.446432+0000","last_peered":"2024-08-18T13:39:39.446432+0000","last_clean":"2024-08-18T13:39:39.446432+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:39:39.446432+0000","last_undegraded":"2024-08-18T13:39:39.446432+0000","last_fullsized":"2024-08-18T13:39:39.446432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427616+0000","last_change":"2024-08-18T13:39:35.413292+0000","last_active":"2024-08-18T13:39:39.427616+0000","last_peered":"2024-08-18T13:39:39.427616+0000","last_clean":"2024-08-18T13:39:39.427616+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:39:39.427616+0000","last_undegraded":"2024-08-18T13:39:39.427616+0000","last_fullsized":"2024-08-18T13:39:39.427616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415812+0000","last_change":"2024-08-18T13:39:35.401196+0000","last_active":"2024-08-18T13:39:39.415812+0000","last_peered":"2024-08-18T13:39:39.415812+0000","last_clean":"2024-08-18T13:39:39.415812+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:39:39.415812+0000","last_undegraded":"2024-08-18T13:39:39.415812+0000","last_fullsized":"2024-08-18T13:39:39.415812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417577+0000","last_change":"2024-08-18T13:39:35.406603+0000","last_active":"2024-08-18T13:39:39.417577+0000","last_peered":"2024-08-18T13:39:39.417577+0000","last_clean":"2024-08-18T13:39:39.417577+0000","last_became_active":"2024-08-18T13:39:35.406331+0000","last_became_peered":"2024-08-18T13:39:35.406331+0000","last_unstale":"2024-08-18T13:39:39.417577+0000","last_undegraded":"2024-08-18T13:39:39.417577+0000","last_fullsized":"2024-08-18T13:39:39.417577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:21:12.232172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429494+0000","last_change":"2024-08-18T13:39:35.404051+0000","last_active":"2024-08-18T13:39:39.429494+0000","last_peered":"2024-08-18T13:39:39.429494+0000","last_clean":"2024-08-18T13:39:39.429494+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:39:39.429494+0000","last_undegraded":"2024-08-18T13:39:39.429494+0000","last_fullsized":"2024-08-18T13:39:39.429494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422959+0000","last_change":"2024-08-18T13:39:35.409288+0000","last_active":"2024-08-18T13:39:39.422959+0000","last_peered":"2024-08-18T13:39:39.422959+0000","last_clean":"2024-08-18T13:39:39.422959+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:39:39.422959+0000","last_undegraded":"2024-08-18T13:39:39.422959+0000","last_fullsized":"2024-08-18T13:39:39.422959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424121+0000","last_change":"2024-08-18T13:39:35.403051+0000","last_active":"2024-08-18T13:39:39.424121+0000","last_peered":"2024-08-18T13:39:39.424121+0000","last_clean":"2024-08-18T13:39:39.424121+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:39:39.424121+0000","last_undegraded":"2024-08-18T13:39:39.424121+0000","last_fullsized":"2024-08-18T13:39:39.424121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430049+0000","last_change":"2024-08-18T13:39:35.412281+0000","last_active":"2024-08-18T13:39:39.430049+0000","last_peered":"2024-08-18T13:39:39.430049+0000","last_clean":"2024-08-18T13:39:39.430049+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:39:39.430049+0000","last_undegraded":"2024-08-18T13:39:39.430049+0000","last_fullsized":"2024-08-18T13:39:39.430049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424466+0000","last_change":"2024-08-18T13:39:35.409220+0000","last_active":"2024-08-18T13:39:39.424466+0000","last_peered":"2024-08-18T13:39:39.424466+0000","last_clean":"2024-08-18T13:39:39.424466+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:39:39.424466+0000","last_undegraded":"2024-08-18T13:39:39.424466+0000","last_fullsized":"2024-08-18T13:39:39.424466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418253+0000","last_change":"2024-08-18T13:39:35.401759+0000","last_active":"2024-08-18T13:39:39.418253+0000","last_peered":"2024-08-18T13:39:39.418253+0000","last_clean":"2024-08-18T13:39:39.418253+0000","last_became_active":"2024-08-18T13:39:35.401476+0000","last_became_peered":"2024-08-18T13:39:35.401476+0000","last_unstale":"2024-08-18T13:39:39.418253+0000","last_undegraded":"2024-08-18T13:39:39.418253+0000","last_fullsized":"2024-08-18T13:39:39.418253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:04:43.997597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423669+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.423669+0000","last_peered":"2024-08-18T13:39:39.423669+0000","last_clean":"2024-08-18T13:39:39.423669+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:39:39.423669+0000","last_undegraded":"2024-08-18T13:39:39.423669+0000","last_fullsized":"2024-08-18T13:39:39.423669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417410+0000","last_change":"2024-08-18T13:39:35.407218+0000","last_active":"2024-08-18T13:39:39.417410+0000","last_peered":"2024-08-18T13:39:39.417410+0000","last_clean":"2024-08-18T13:39:39.417410+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:39:39.417410+0000","last_undegraded":"2024-08-18T13:39:39.417410+0000","last_fullsized":"2024-08-18T13:39:39.417410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429387+0000","last_change":"2024-08-18T13:39:35.407339+0000","last_active":"2024-08-18T13:39:39.429387+0000","last_peered":"2024-08-18T13:39:39.429387+0000","last_clean":"2024-08-18T13:39:39.429387+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:39:39.429387+0000","last_undegraded":"2024-08-18T13:39:39.429387+0000","last_fullsized":"2024-08-18T13:39:39.429387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429062+0000","last_change":"2024-08-18T13:39:35.408647+0000","last_active":"2024-08-18T13:39:39.429062+0000","last_peered":"2024-08-18T13:39:39.429062+0000","last_clean":"2024-08-18T13:39:39.429062+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:39:39.429062+0000","last_undegraded":"2024-08-18T13:39:39.429062+0000","last_fullsized":"2024-08-18T13:39:39.429062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450837+0000","last_change":"2024-08-18T13:39:35.418000+0000","last_active":"2024-08-18T13:39:39.450837+0000","last_peered":"2024-08-18T13:39:39.450837+0000","last_clean":"2024-08-18T13:39:39.450837+0000","last_became_active":"2024-08-18T13:39:35.417661+0000","last_became_peered":"2024-08-18T13:39:35.417661+0000","last_unstale":"2024-08-18T13:39:39.450837+0000","last_undegraded":"2024-08-18T13:39:39.450837+0000","last_fullsized":"2024-08-18T13:39:39.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:53:44.739585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416124+0000","last_change":"2024-08-18T13:39:35.405425+0000","last_active":"2024-08-18T13:39:39.416124+0000","last_peered":"2024-08-18T13:39:39.416124+0000","last_clean":"2024-08-18T13:39:39.416124+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:39:39.416124+0000","last_undegraded":"2024-08-18T13:39:39.416124+0000","last_fullsized":"2024-08-18T13:39:39.416124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423886+0000","last_change":"2024-08-18T13:39:35.400889+0000","last_active":"2024-08-18T13:39:39.423886+0000","last_peered":"2024-08-18T13:39:39.423886+0000","last_clean":"2024-08-18T13:39:39.423886+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:39:39.423886+0000","last_undegraded":"2024-08-18T13:39:39.423886+0000","last_fullsized":"2024-08-18T13:39:39.423886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418057+0000","last_change":"2024-08-18T13:39:35.407634+0000","last_active":"2024-08-18T13:39:39.418057+0000","last_peered":"2024-08-18T13:39:39.418057+0000","last_clean":"2024-08-18T13:39:39.418057+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:39:39.418057+0000","last_undegraded":"2024-08-18T13:39:39.418057+0000","last_fullsized":"2024-08-18T13:39:39.418057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422473+0000","last_change":"2024-08-18T13:39:35.397509+0000","last_active":"2024-08-18T13:39:39.422473+0000","last_peered":"2024-08-18T13:39:39.422473+0000","last_clean":"2024-08-18T13:39:39.422473+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:39:39.422473+0000","last_undegraded":"2024-08-18T13:39:39.422473+0000","last_fullsized":"2024-08-18T13:39:39.422473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425330+0000","last_change":"2024-08-18T13:39:35.404533+0000","last_active":"2024-08-18T13:39:39.425330+0000","last_peered":"2024-08-18T13:39:39.425330+0000","last_clean":"2024-08-18T13:39:39.425330+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:39:39.425330+0000","last_undegraded":"2024-08-18T13:39:39.425330+0000","last_fullsized":"2024-08-18T13:39:39.425330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425567+0000","last_change":"2024-08-18T13:39:35.404839+0000","last_active":"2024-08-18T13:39:39.425567+0000","last_peered":"2024-08-18T13:39:39.425567+0000","last_clean":"2024-08-18T13:39:39.425567+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:39:39.425567+0000","last_undegraded":"2024-08-18T13:39:39.425567+0000","last_fullsized":"2024-08-18T13:39:39.425567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421633+0000","last_change":"2024-08-18T13:39:35.405867+0000","last_active":"2024-08-18T13:39:39.421633+0000","last_peered":"2024-08-18T13:39:39.421633+0000","last_clean":"2024-08-18T13:39:39.421633+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:39:39.421633+0000","last_undegraded":"2024-08-18T13:39:39.421633+0000","last_fullsized":"2024-08-18T13:39:39.421633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429697+0000","last_change":"2024-08-18T13:39:35.411173+0000","last_active":"2024-08-18T13:39:39.429697+0000","last_peered":"2024-08-18T13:39:39.429697+0000","last_clean":"2024-08-18T13:39:39.429697+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:39:39.429697+0000","last_undegraded":"2024-08-18T13:39:39.429697+0000","last_fullsized":"2024-08-18T13:39:39.429697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429962+0000","last_change":"2024-08-18T13:39:35.418013+0000","last_active":"2024-08-18T13:39:39.429962+0000","last_peered":"2024-08-18T13:39:39.429962+0000","last_clean":"2024-08-18T13:39:39.429962+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:39:39.429962+0000","last_undegraded":"2024-08-18T13:39:39.429962+0000","last_fullsized":"2024-08-18T13:39:39.429962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450803+0000","last_change":"2024-08-18T13:39:35.421633+0000","last_active":"2024-08-18T13:39:39.450803+0000","last_peered":"2024-08-18T13:39:39.450803+0000","last_clean":"2024-08-18T13:39:39.450803+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:39:39.450803+0000","last_undegraded":"2024-08-18T13:39:39.450803+0000","last_fullsized":"2024-08-18T13:39:39.450803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425234+0000","last_change":"2024-08-18T13:39:35.411443+0000","last_active":"2024-08-18T13:39:39.425234+0000","last_peered":"2024-08-18T13:39:39.425234+0000","last_clean":"2024-08-18T13:39:39.425234+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:39:39.425234+0000","last_undegraded":"2024-08-18T13:39:39.425234+0000","last_fullsized":"2024-08-18T13:39:39.425234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450366+0000","last_change":"2024-08-18T13:39:35.415548+0000","last_active":"2024-08-18T13:39:39.450366+0000","last_peered":"2024-08-18T13:39:39.450366+0000","last_clean":"2024-08-18T13:39:39.450366+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:39:39.450366+0000","last_undegraded":"2024-08-18T13:39:39.450366+0000","last_fullsized":"2024-08-18T13:39:39.450366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417246+0000","last_change":"2024-08-18T13:39:35.410795+0000","last_active":"2024-08-18T13:39:39.417246+0000","last_peered":"2024-08-18T13:39:39.417246+0000","last_clean":"2024-08-18T13:39:39.417246+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:39:39.417246+0000","last_undegraded":"2024-08-18T13:39:39.417246+0000","last_fullsized":"2024-08-18T13:39:39.417246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450710+0000","last_change":"2024-08-18T13:39:35.417550+0000","last_active":"2024-08-18T13:39:39.450710+0000","last_peered":"2024-08-18T13:39:39.450710+0000","last_clean":"2024-08-18T13:39:39.450710+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:39:39.450710+0000","last_undegraded":"2024-08-18T13:39:39.450710+0000","last_fullsized":"2024-08-18T13:39:39.450710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422555+0000","last_change":"2024-08-18T13:39:35.410002+0000","last_active":"2024-08-18T13:39:39.422555+0000","last_peered":"2024-08-18T13:39:39.422555+0000","last_clean":"2024-08-18T13:39:39.422555+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:39:39.422555+0000","last_undegraded":"2024-08-18T13:39:39.422555+0000","last_fullsized":"2024-08-18T13:39:39.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427751+0000","last_change":"2024-08-18T13:39:35.406627+0000","last_active":"2024-08-18T13:39:39.427751+0000","last_peered":"2024-08-18T13:39:39.427751+0000","last_clean":"2024-08-18T13:39:39.427751+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:39:39.427751+0000","last_undegraded":"2024-08-18T13:39:39.427751+0000","last_fullsized":"2024-08-18T13:39:39.427751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423078+0000","last_change":"2024-08-18T13:39:35.405750+0000","last_active":"2024-08-18T13:39:39.423078+0000","last_peered":"2024-08-18T13:39:39.423078+0000","last_clean":"2024-08-18T13:39:39.423078+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:39:39.423078+0000","last_undegraded":"2024-08-18T13:39:39.423078+0000","last_fullsized":"2024-08-18T13:39:39.423078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450591+0000","last_change":"2024-08-18T13:39:35.417195+0000","last_active":"2024-08-18T13:39:39.450591+0000","last_peered":"2024-08-18T13:39:39.450591+0000","last_clean":"2024-08-18T13:39:39.450591+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:39:39.450591+0000","last_undegraded":"2024-08-18T13:39:39.450591+0000","last_fullsized":"2024-08-18T13:39:39.450591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4548,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":856064,"data_stored":693001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]}]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":270336,"data_stored":102189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":258048,"data_stored":102063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":102223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]}]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":864256,"data_stored":693057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":262144,"data_stored":102153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:39:58.835 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-18T13:39:58.835 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-18T13:39:58.835 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-18T13:39:58.835 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-18T13:39:58.836 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-18T13:39:58.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.938+0000 7f5f21f19700 1 Processor -- start 2024-08-18T13:39:58.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.938+0000 7f5f21f19700 1 -- start start 2024-08-18T13:39:58.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5f1c0684e0 0x7f5f1c155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0c1110 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0c1cb0 con 0x7f5f1c1556f0 2024-08-18T13:39:58.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c0b90 con 0x7f5f1c0684e0 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f13fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5f1c0684e0 0x7f5f1c155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f13fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5f1c0684e0 0x7f5f1c155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44992/0 (socket says 172.21.15.118:44992) 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f13fff700 1 -- 172.21.15.118:0/825614220 learned_addr learned my addr 172.21.15.118:0/825614220 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f13fff700 1 -- 172.21.15.118:0/825614220 >> v1:172.21.15.178:6789/0 conn(0x7f5f1c1556f0 legacy=0x7f5f1c0c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f13fff700 1 -- 172.21.15.118:0/825614220 >> v1:172.21.15.118:6789/0 conn(0x7f5f1c0be4b0 legacy=0x7f5f1c151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f13fff700 1 -- 172.21.15.118:0/825614220 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0c8060 con 0x7f5f1c0684e0 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f13fff700 1 --2- 172.21.15.118:0/825614220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0684e0 0x7f5f1c155ac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0099b0 tx=0x7f5f0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc60041655dc248b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f137fe700 1 -- 172.21.15.118:0/825614220 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00b820 con 0x7f5f1c0684e0 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f137fe700 1 -- 172.21.15.118:0/825614220 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c00b980 con 0x7f5f1c0684e0 2024-08-18T13:39:58.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f137fe700 1 -- 172.21.15.118:0/825614220 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00bd00 con 0x7f5f1c0684e0 2024-08-18T13:39:58.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- 172.21.15.118:0/825614220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0684e0 msgr2=0x7f5f1c155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 --2- 172.21.15.118:0/825614220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0684e0 0x7f5f1c155ac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0099b0 tx=0x7f5f0c009250 comp rx=0 tx=0).stop 2024-08-18T13:39:58.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- 172.21.15.118:0/825614220 shutdown_connections 2024-08-18T13:39:58.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 --2- 172.21.15.118:0/825614220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0684e0 0x7f5f1c155ac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- 172.21.15.118:0/825614220 >> 172.21.15.118:0/825614220 conn(0x7f5f1c06c020 msgr2=0x7f5f1c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:58.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- 172.21.15.118:0/825614220 shutdown_connections 2024-08-18T13:39:58.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.942+0000 7f5f21f19700 1 -- 172.21.15.118:0/825614220 wait complete. 2024-08-18T13:39:58.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 Processor -- start 2024-08-18T13:39:58.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 -- start start 2024-08-18T13:39:58.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0684e0 0x7f5f1c149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0be4b0 0x7f5f1c1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f1c1556f0 0x7f5f1c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c7110 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c14d740 con 0x7f5f1c0684e0 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c7340 con 0x7f5f1c1556f0 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0be4b0 0x7f5f1c1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0be4b0 0x7f5f1c1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44996/0 (socket says 172.21.15.118:44996) 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 -- 172.21.15.118:0/4056665034 learned_addr learned my addr 172.21.15.118:0/4056665034 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f20f17700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0684e0 0x7f5f1c149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21718700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f1c1556f0 0x7f5f1c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 -- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f1c1556f0 msgr2=0x7f5f1c157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f1c1556f0 0x7f5f1c157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 -- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0684e0 msgr2=0x7f5f1c149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0684e0 0x7f5f1c149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 -- 172.21.15.118:0/4056665034 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0094f0 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f13fff700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0be4b0 0x7f5f1c1575b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0090b0 tx=0x7f5f0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f10ff9700 1 -- 172.21.15.118:0/4056665034 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c010d80 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f10ff9700 1 -- 172.21.15.118:0/4056665034 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c018420 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c14f1f0 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f10ff9700 1 -- 172.21.15.118:0/4056665034 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0187e0 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.946+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c156bf0 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.950+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c06efe0 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.950+0000 7f5f10ff9700 1 -- 172.21.15.118:0/4056665034 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5f0c018bd0 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.950+0000 7f5f10ff9700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f08060860 0x7f5f08062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:58.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.950+0000 7f5f10ff9700 1 -- 172.21.15.118:0/4056665034 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f5f0c014070 con 0x7f5f1c0be4b0 2024-08-18T13:39:58.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.950+0000 7f5f20f17700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f08060860 0x7f5f08062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:58.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.950+0000 7f5f20f17700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f08060860 0x7f5f08062d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f18003a30 tx=0x7f5f18007ea0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:58.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:58.958+0000 7f5f10ff9700 1 -- 172.21.15.118:0/4056665034 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5f1c06efe0 con 0x7f5f1c0be4b0 2024-08-18T13:39:59.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.158+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5f1c06efe0 con 0x7f5f1c0be4b0 2024-08-18T13:39:59.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f10ff9700 1 -- 172.21.15.118:0/4056665034 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154539 (secure 0 0 0) 0x7f5f1c06efe0 con 0x7f5f1c0be4b0 2024-08-18T13:39:59.167 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:59.168 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f08060860 msgr2=0x7f5f08062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f08060860 0x7f5f08062d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f18003a30 tx=0x7f5f18007ea0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0be4b0 msgr2=0x7f5f1c1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0be4b0 0x7f5f1c1575b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0090b0 tx=0x7f5f0c018000 comp rx=0 tx=0).stop 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 shutdown_connections 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f08060860 0x7f5f08062d10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f1c1556f0 0x7f5f1c157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f1c0be4b0 0x7f5f1c1575b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 --2- 172.21.15.118:0/4056665034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f1c0684e0 0x7f5f1c149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 >> 172.21.15.118:0/4056665034 conn(0x7f5f1c06c020 msgr2=0x7f5f1c144730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 shutdown_connections 2024-08-18T13:39:59.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.162+0000 7f5f21f19700 1 -- 172.21.15.118:0/4056665034 wait complete. 2024-08-18T13:39:59.181 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":24192},{"type":"v1","addr":"172.21.15.118:6833","nonce":24192}]},"active_addr":"172.21.15.118:6833/24192","active_change":"2024-08-18T13:39:15.178973+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":266408872}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":122515762}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":2879618348}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":924809659}]}]} 2024-08-18T13:39:59.184 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-18T13:39:59.184 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-18T13:39:59.184 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:59.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee74830700 1 Processor -- start 2024-08-18T13:39:59.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee74830700 1 -- start start 2024-08-18T13:39:59.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee74830700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fee700aeba0 0x7fee70155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee74830700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee700c7120 0x7fee7014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee74830700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee700b0ee0 con 0x7fee700abaf0 2024-08-18T13:39:59.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee74830700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee700b1180 con 0x7fee700aeba0 2024-08-18T13:39:59.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee74830700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee700b1420 con 0x7fee700c7120 2024-08-18T13:39:59.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee6e59c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fee700abaf0 0x7fee700c6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45870/0 (socket says 172.21.15.118:45870) 2024-08-18T13:39:59.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.282+0000 7fee6e59c700 1 -- 172.21.15.118:0/141868816 learned_addr learned my addr 172.21.15.118:0/141868816 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:59.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6ed9d700 1 --2- 172.21.15.118:0/141868816 >> v2:172.21.15.178:3300/0 conn(0x7fee700aeba0 0x7fee70155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6f59e700 1 --2- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee700c7120 0x7fee7014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6dd9b700 1 -- 172.21.15.118:0/141868816 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507593328 0 0) 0x7fee700b0ee0 con 0x7fee700abaf0 2024-08-18T13:39:59.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6dd9b700 1 -- 172.21.15.118:0/141868816 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee58003570 con 0x7fee700abaf0 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6ed9d700 1 -- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee700c7120 msgr2=0x7fee7014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6ed9d700 1 --2- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee700c7120 0x7fee7014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6ed9d700 1 -- 172.21.15.118:0/141868816 >> v1:172.21.15.118:6789/0 conn(0x7fee700abaf0 legacy=0x7fee700c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6ed9d700 1 -- 172.21.15.118:0/141868816 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee700b16c0 con 0x7fee700aeba0 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6ed9d700 1 --2- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700aeba0 0x7fee70155160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee640098e0 tx=0x7fee640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce28de288182552a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6dd9b700 1 -- 172.21.15.118:0/141868816 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64011630 con 0x7fee700aeba0 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6dd9b700 1 -- 172.21.15.118:0/141868816 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee64011790 con 0x7fee700aeba0 2024-08-18T13:39:59.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee6dd9b700 1 -- 172.21.15.118:0/141868816 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64011b30 con 0x7fee700aeba0 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 -- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700aeba0 msgr2=0x7fee70155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 --2- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700aeba0 0x7fee70155160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee640098e0 tx=0x7fee640091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 -- 172.21.15.118:0/141868816 shutdown_connections 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 --2- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee700c7120 0x7fee7014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 --2- 172.21.15.118:0/141868816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700aeba0 0x7fee70155160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 -- 172.21.15.118:0/141868816 >> 172.21.15.118:0/141868816 conn(0x7fee700cbb80 msgr2=0x7fee700cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 -- 172.21.15.118:0/141868816 shutdown_connections 2024-08-18T13:39:59.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.286+0000 7fee74830700 1 -- 172.21.15.118:0/141868816 wait complete. 2024-08-18T13:39:59.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 Processor -- start 2024-08-18T13:39:59.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 -- start start 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700aeba0 0x7fee70165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700c7120 0x7fee7014eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee70156850 0x7fee70156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70168720 con 0x7fee700aeba0 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee701563c0 con 0x7fee700c7120 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee74830700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee70157210 con 0x7fee70156850 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee6ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700aeba0 0x7fee70165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee6ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700aeba0 0x7fee70165180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45006/0 (socket says 172.21.15.118:45006) 2024-08-18T13:39:59.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.290+0000 7fee6ed9d700 1 -- 172.21.15.118:0/3436486444 learned_addr learned my addr 172.21.15.118:0/3436486444 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6e59c700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700c7120 0x7fee7014eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6f59e700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee70156850 0x7fee70156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6ed9d700 1 -- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee70156850 msgr2=0x7fee70156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6ed9d700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee70156850 0x7fee70156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6ed9d700 1 -- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700c7120 msgr2=0x7fee7014eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6ed9d700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700c7120 0x7fee7014eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6ed9d700 1 -- 172.21.15.118:0/3436486444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64009420 con 0x7fee700aeba0 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee6ed9d700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700aeba0 0x7fee70165180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ed0 tx=0x7fee64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee56ffd700 1 -- 172.21.15.118:0/3436486444 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64011900 con 0x7fee700aeba0 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee56ffd700 1 -- 172.21.15.118:0/3436486444 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee64011a60 con 0x7fee700aeba0 2024-08-18T13:39:59.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee70168950 con 0x7fee700aeba0 2024-08-18T13:39:59.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee56ffd700 1 -- 172.21.15.118:0/3436486444 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64023620 con 0x7fee700aeba0 2024-08-18T13:39:59.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee70168f10 con 0x7fee700aeba0 2024-08-18T13:39:59.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee700713a0 con 0x7fee700aeba0 2024-08-18T13:39:59.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.294+0000 7fee56ffd700 1 -- 172.21.15.118:0/3436486444 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fee6400b400 con 0x7fee700aeba0 2024-08-18T13:39:59.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.298+0000 7fee56ffd700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fee5c060860 0x7fee5c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.298+0000 7fee56ffd700 1 -- 172.21.15.118:0/3436486444 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fee6407e2f0 con 0x7fee700aeba0 2024-08-18T13:39:59.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.298+0000 7fee6e59c700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fee5c060860 0x7fee5c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.298+0000 7fee6e59c700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fee5c060860 0x7fee5c062d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee700c6100 tx=0x7fee60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.306+0000 7fee56ffd700 1 -- 172.21.15.118:0/3436486444 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fee6404e450 con 0x7fee700aeba0 2024-08-18T13:39:59.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.470+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee700c78b0 con 0x7fee700aeba0 2024-08-18T13:39:59.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee56ffd700 1 -- 172.21.15.118:0/3436486444 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7fee64051a70 con 0x7fee700aeba0 2024-08-18T13:39:59.479 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:59.479 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fee5c060860 msgr2=0x7fee5c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fee5c060860 0x7fee5c062d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee700c6100 tx=0x7fee60009040 comp rx=0 tx=0).stop 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700aeba0 msgr2=0x7fee70165180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700aeba0 0x7fee70165180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ed0 tx=0x7fee64017000 comp rx=0 tx=0).stop 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 shutdown_connections 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fee5c060860 0x7fee5c062d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee70156850 0x7fee70156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee700c7120 0x7fee7014eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 --2- 172.21.15.118:0/3436486444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee700aeba0 0x7fee70165180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 >> 172.21.15.118:0/3436486444 conn(0x7fee700cbb80 msgr2=0x7fee701550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 shutdown_connections 2024-08-18T13:39:59.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.474+0000 7fee74830700 1 -- 172.21.15.118:0/3436486444 wait complete. 2024-08-18T13:39:59.493 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-18T13:39:59.493 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:39:59.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f017483e700 1 Processor -- start 2024-08-18T13:39:59.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f017483e700 1 -- start start 2024-08-18T13:39:59.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f017483e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f0170152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f017483e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f01700c0c50 0x7f01700c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f017483e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01700c1610 con 0x7f017005e6b0 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f017483e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0170068370 con 0x7f01700c0c50 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f017483e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0170152d80 con 0x7f017014bf60 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f016f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f01700c0c50 0x7f01700c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f016ed9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f0170152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f016e59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f017005e6b0 0x7f017006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:33942/0 (socket says 172.21.15.118:33942) 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f016f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f01700c0c50 0x7f01700c10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45018/0 (socket says 172.21.15.118:45018) 2024-08-18T13:39:59.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.590+0000 7f016e59c700 1 -- 172.21.15.118:0/2693034741 learned_addr learned my addr 172.21.15.118:0/2693034741 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:59.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016f59e700 1 -- 172.21.15.118:0/2693034741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 msgr2=0x7f0170152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016f59e700 1 --2- 172.21.15.118:0/2693034741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f0170152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016f59e700 1 -- 172.21.15.118:0/2693034741 >> v1:172.21.15.178:6789/0 conn(0x7f017005e6b0 legacy=0x7f017006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016dd9b700 1 -- 172.21.15.118:0/2693034741 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865697574 0 0) 0x7f01700c1610 con 0x7f017005e6b0 2024-08-18T13:39:59.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016f59e700 1 -- 172.21.15.118:0/2693034741 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0170067470 con 0x7f01700c0c50 2024-08-18T13:39:59.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016f59e700 1 --2- 172.21.15.118:0/2693034741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01700c10d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0168009e60 tx=0x7f0168009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b335aba6470b97 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016dd9b700 1 -- 172.21.15.118:0/2693034741 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016800e070 con 0x7f01700c0c50 2024-08-18T13:39:59.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016dd9b700 1 -- 172.21.15.118:0/2693034741 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016800bbf0 con 0x7f01700c0c50 2024-08-18T13:39:59.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f016dd9b700 1 -- 172.21.15.118:0/2693034741 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168012430 con 0x7f01700c0c50 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 -- 172.21.15.118:0/2693034741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 msgr2=0x7f01700c10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 --2- 172.21.15.118:0/2693034741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01700c10d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0168009e60 tx=0x7f0168009780 comp rx=0 tx=0).stop 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 -- 172.21.15.118:0/2693034741 shutdown_connections 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 --2- 172.21.15.118:0/2693034741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01700c10d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 --2- 172.21.15.118:0/2693034741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f0170152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 -- 172.21.15.118:0/2693034741 >> 172.21.15.118:0/2693034741 conn(0x7f01700c81e0 msgr2=0x7f01700c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 -- 172.21.15.118:0/2693034741 shutdown_connections 2024-08-18T13:39:59.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.594+0000 7f017483e700 1 -- 172.21.15.118:0/2693034741 wait complete. 2024-08-18T13:39:59.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 Processor -- start 2024-08-18T13:39:59.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 -- start start 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017005e6b0 0x7f0170145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01701485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f017013eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0170145e10 con 0x7f01700c0c50 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0170149b30 con 0x7f017005e6b0 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017014a860 con 0x7f017014bf60 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01701485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01701485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45030/0 (socket says 172.21.15.118:45030) 2024-08-18T13:39:59.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 -- 172.21.15.118:0/3531563102 learned_addr learned my addr 172.21.15.118:0/3531563102 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016f59e700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f017013eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016ed9d700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017005e6b0 0x7f0170145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 -- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 msgr2=0x7f017013eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f017013eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 -- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017005e6b0 msgr2=0x7f0170145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017005e6b0 0x7f0170145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 -- 172.21.15.118:0/3531563102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168009a20 con 0x7f01700c0c50 2024-08-18T13:39:59.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f016e59c700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01701485e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f016400c7e0 tx=0x7f0164007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f0156ffd700 1 -- 172.21.15.118:0/3531563102 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164005b30 con 0x7f01700c0c50 2024-08-18T13:39:59.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f0156ffd700 1 -- 172.21.15.118:0/3531563102 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164005c90 con 0x7f01700c0c50 2024-08-18T13:39:59.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f0156ffd700 1 -- 172.21.15.118:0/3531563102 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164007300 con 0x7f01700c0c50 2024-08-18T13:39:59.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017006abb0 con 0x7f01700c0c50 2024-08-18T13:39:59.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.598+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0170144aa0 con 0x7f01700c0c50 2024-08-18T13:39:59.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.602+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01700713a0 con 0x7f01700c0c50 2024-08-18T13:39:59.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.602+0000 7f0156ffd700 1 -- 172.21.15.118:0/3531563102 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f016401b3b0 con 0x7f01700c0c50 2024-08-18T13:39:59.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.602+0000 7f0156ffd700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f015c060890 0x7f015c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.602+0000 7f016ed9d700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f015c060890 0x7f015c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.602+0000 7f0156ffd700 1 -- 172.21.15.118:0/3531563102 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f016407c340 con 0x7f01700c0c50 2024-08-18T13:39:59.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.602+0000 7f016ed9d700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f015c060890 0x7f015c062d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0160009d10 tx=0x7f01600091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.610+0000 7f0156ffd700 1 -- 172.21.15.118:0/3531563102 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f016404c4a0 con 0x7f01700c0c50 2024-08-18T13:39:59.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.778+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f017005ecc0 con 0x7f01700c0c50 2024-08-18T13:39:59.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f0156ffd700 1 -- 172.21.15.118:0/3531563102 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f0164001d60 con 0x7f01700c0c50 2024-08-18T13:39:59.786 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:39:59.786 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:39:59.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f015c060890 msgr2=0x7f015c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f015c060890 0x7f015c062d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0160009d10 tx=0x7f01600091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 msgr2=0x7f01701485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01701485e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f016400c7e0 tx=0x7f0164007df0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 shutdown_connections 2024-08-18T13:39:59.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f015c060890 0x7f015c062d40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f017014bf60 0x7f017013eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f01700c0c50 0x7f01701485e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 --2- 172.21.15.118:0/3531563102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017005e6b0 0x7f0170145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 >> 172.21.15.118:0/3531563102 conn(0x7f01700c81e0 msgr2=0x7f017006a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 shutdown_connections 2024-08-18T13:39:59.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.782+0000 7f017483e700 1 -- 172.21.15.118:0/3531563102 wait complete. 2024-08-18T13:39:59.801 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-18T13:39:59.801 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-18T13:39:59.801 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-18T13:39:59.802 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-18T13:39:59.802 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-18T13:39:59.803 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-18T13:39:59.803 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-08-18T13:39:59.804 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-08-18T13:39:59.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.934+0000 7fcc9ddd7700 1 Processor -- start 2024-08-18T13:39:59.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.934+0000 7fcc9ddd7700 1 -- start start 2024-08-18T13:39:59.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.934+0000 7f93a3fff700 1 Processor -- start 2024-08-18T13:39:59.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.934+0000 7f93a3fff700 1 -- start start 2024-08-18T13:39:59.941 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9ddd7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcc98153690 0x7fcc980673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9ddd7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9806ee60 con 0x7fcc9806f2f0 2024-08-18T13:39:59.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9ddd7700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9806f730 con 0x7fcc980c8200 2024-08-18T13:39:59.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9ddd7700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc9815a640 con 0x7fcc98153690 2024-08-18T13:39:59.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fcc9806f2f0 0x7fcc980c29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45884/0 (socket says 172.21.15.118:45884) 2024-08-18T13:39:59.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 -- 172.21.15.118:0/217583087 learned_addr learned my addr 172.21.15.118:0/217583087 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:59.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a3fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f93a4153690 0x7f93a40673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a3fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a406f2f0 0x7f93a40c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a3fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a406f770 con 0x7f93a40c8200 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a3fff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a415a640 con 0x7f93a4153690 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a3fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40c85d0 con 0x7f93a406f2f0 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a27fc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f93a40c8200 0x7f93a40c22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45890/0 (socket says 172.21.15.118:45890) 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a27fc700 1 -- 172.21.15.118:0/2834439634 learned_addr learned my addr 172.21.15.118:0/2834439634 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804951723 0 0) 0x7fcc9806ee60 con 0x7fcc9806f2f0 2024-08-18T13:39:59.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc80003570 con 0x7fcc9806f2f0 2024-08-18T13:39:59.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a2ffd700 1 --2- 172.21.15.118:0/2834439634 >> v2:172.21.15.178:3300/0 conn(0x7f93a4153690 0x7f93a40673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268875879 0 0) 0x7fcc9806f730 con 0x7fcc980c8200 2024-08-18T13:39:59.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9cdd5700 1 --2- 172.21.15.118:0/217583087 >> v2:172.21.15.178:3300/0 conn(0x7fcc98153690 0x7fcc980673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc9806ee60 con 0x7fcc980c8200 2024-08-18T13:39:59.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a37fe700 1 --2- 172.21.15.118:0/2834439634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a406f2f0 0x7f93a40c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2476130604 0 0) 0x7fcc80003570 con 0x7fcc9806f2f0 2024-08-18T13:39:59.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 >> v2:172.21.15.178:3300/0 conn(0x7fcc98153690 msgr2=0x7fcc980673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 --2- 172.21.15.118:0/217583087 >> v2:172.21.15.178:3300/0 conn(0x7fcc98153690 0x7fcc980673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 >> v1:172.21.15.178:6789/0 conn(0x7fcc980c8200 legacy=0x7fcc980c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc98153d10 con 0x7fcc9806f2f0 2024-08-18T13:39:59.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fcc88001470 con 0x7fcc9806f2f0 2024-08-18T13:39:59.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 >> v1:172.21.15.118:6789/0 conn(0x7fcc9806f2f0 legacy=0x7fcc980c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc80003c40 0x7fcc80004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc80004810 0x7fcc80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc800053c0 0x7fcc8000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc800053c0 0x7fcc8000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9815a640 con 0x7fcc800053c0 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8000c300 con 0x7fcc80004810 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8000c4c0 con 0x7fcc80003c40 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a37fe700 1 -- 172.21.15.118:0/2834439634 >> v2:172.21.15.178:3300/0 conn(0x7f93a4153690 msgr2=0x7f93a40673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a37fe700 1 --2- 172.21.15.118:0/2834439634 >> v2:172.21.15.178:3300/0 conn(0x7f93a4153690 0x7f93a40673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a37fe700 1 -- 172.21.15.118:0/2834439634 >> v1:172.21.15.118:6789/0 conn(0x7f93a40c8200 legacy=0x7f93a40c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9cdd5700 1 --2- 172.21.15.118:0/217583087 >> v2:172.21.15.178:3300/0 conn(0x7fcc98153690 0x7fcc980673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:39:59.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc97fff700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc80004810 0x7fcc80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a37fe700 1 -- 172.21.15.118:0/2834439634 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a406ee60 con 0x7f93a406f2f0 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9cdd5700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc80003c40 0x7fcc80004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 -- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc80003c40 msgr2=0x7fcc80004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc80003c40 0x7fcc80004010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 -- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc80004810 msgr2=0x7fcc80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc80004810 0x7fcc80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 -- 172.21.15.118:0/217583087 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80003570 con 0x7fcc800053c0 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 -- 172.21.15.118:0/217583087 reap_dead start 2024-08-18T13:39:59.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a37fe700 1 --2- 172.21.15.118:0/2834439634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a406f2f0 0x7f93a40c29b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f939c009e60 tx=0x7f939c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9136cad103a3bf8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc9d5d6700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc800053c0 0x7fcc8000ba60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800a810 tx=0x7fcc88007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d1645280dddf63 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88005920 con 0x7fcc800053c0 2024-08-18T13:39:59.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88005aa0 con 0x7fcc800053c0 2024-08-18T13:39:59.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7fcc977fe700 1 -- 172.21.15.118:0/217583087 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88005e60 con 0x7fcc800053c0 2024-08-18T13:39:59.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a1ffb700 1 -- 172.21.15.118:0/2834439634 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00e070 con 0x7f93a406f2f0 2024-08-18T13:39:59.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.938+0000 7f93a1ffb700 1 -- 172.21.15.118:0/2834439634 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c00ba60 con 0x7f93a406f2f0 2024-08-18T13:39:59.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a1ffb700 1 -- 172.21.15.118:0/2834439634 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00be00 con 0x7f93a406f2f0 2024-08-18T13:39:59.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/217583087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc800053c0 msgr2=0x7fcc8000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc800053c0 0x7fcc8000ba60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800a810 tx=0x7fcc88007dc0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/217583087 shutdown_connections 2024-08-18T13:39:59.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 --2- 172.21.15.118:0/217583087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc800053c0 0x7fcc8000ba60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/217583087 >> 172.21.15.118:0/217583087 conn(0x7fcc9806c020 msgr2=0x7fcc980c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 -- 172.21.15.118:0/2834439634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a406f2f0 msgr2=0x7f93a40c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 --2- 172.21.15.118:0/2834439634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a406f2f0 0x7f93a40c29b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f939c009e60 tx=0x7f939c009780 comp rx=0 tx=0).stop 2024-08-18T13:39:59.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/217583087 shutdown_connections 2024-08-18T13:39:59.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/217583087 wait complete. 2024-08-18T13:39:59.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 -- 172.21.15.118:0/2834439634 shutdown_connections 2024-08-18T13:39:59.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 --2- 172.21.15.118:0/2834439634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a406f2f0 0x7f93a40c29b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 --2- 172.21.15.118:0/2834439634 >> v2:172.21.15.178:3300/0 conn(0x7f93a4153690 0x7f93a40673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 -- 172.21.15.118:0/2834439634 >> 172.21.15.118:0/2834439634 conn(0x7f93a406c020 msgr2=0x7f93a40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f1ef5eb6700 1 Processor -- start 2024-08-18T13:39:59.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 -- 172.21.15.118:0/2834439634 shutdown_connections 2024-08-18T13:39:59.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 -- 172.21.15.118:0/2834439634 wait complete. 2024-08-18T13:39:59.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f1ef5eb6700 1 -- start start 2024-08-18T13:39:59.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 Processor -- start 2024-08-18T13:39:59.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 -- start start 2024-08-18T13:39:59.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 Processor -- start 2024-08-18T13:39:59.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7fcc9ddd7700 1 -- start start 2024-08-18T13:39:59.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f87594d6700 1 Processor -- start 2024-08-18T13:39:59.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8bf815c050 0x7f8bf80673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf80c8290 0x7f8bf80c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8bf8161e70 0x7f8bf80c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80c33e0 con 0x7f8bf815c050 2024-08-18T13:39:59.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf806f1c0 con 0x7f8bf8161e70 2024-08-18T13:39:59.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf7fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf806cf00 con 0x7f8bf80c8290 2024-08-18T13:39:59.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f1ef5eb6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1ef00c1c00 0x7f1ef00c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f87594d6700 1 -- start start 2024-08-18T13:39:59.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f5d7ed5d700 1 Processor -- start 2024-08-18T13:39:59.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f93a3fff700 1 Processor -- start 2024-08-18T13:39:59.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f1eeeffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1ef00c1c00 0x7f1ef00c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f1eef7fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f1ef00c1830 0x7f1ef00673d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45902/0 (socket says 172.21.15.118:45902) 2024-08-18T13:39:59.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f1eeeffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1ef00c1c00 0x7f1ef00c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45052/0 (socket says 172.21.15.118:45052) 2024-08-18T13:39:59.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf67fc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf80c8290 0x7f8bf80c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.942+0000 7f8bf6ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8bf815c050 0x7f8bf80673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc800053c0 0x7fcc9815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf6ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8bf815c050 0x7f8bf80673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45042/0 (socket says 172.21.15.118:45042) 2024-08-18T13:39:59.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf6ffd700 1 -- 172.21.15.118:0/324346889 learned_addr learned my addr 172.21.15.118:0/324346889 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf77fe700 1 --2- 172.21.15.118:0/324346889 >> v2:172.21.15.178:3300/0 conn(0x7f8bf8161e70 0x7f8bf80c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc980c7100 0x7fcc980bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc9814dfc0 0x7fcc9814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980c85d0 con 0x7fcc980c7100 2024-08-18T13:39:59.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98159870 con 0x7fcc9814dfc0 2024-08-18T13:39:59.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980c7530 con 0x7fcc800053c0 2024-08-18T13:39:59.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eef7fe700 1 -- 172.21.15.118:0/4116874991 learned_addr learned my addr 172.21.15.118:0/4116874991 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:39:59.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef00c72e0 con 0x7f1ef00c1830 2024-08-18T13:39:59.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc97fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc980c7100 0x7fcc980bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc97fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc980c7100 0x7fcc980bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45064/0 (socket says 172.21.15.118:45064) 2024-08-18T13:39:59.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc97fff700 1 -- 172.21.15.118:0/1576157988 learned_addr learned my addr 172.21.15.118:0/1576157988 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef0155e60 con 0x7f1ef00c23b0 2024-08-18T13:39:59.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef01563c0 con 0x7f1ef00c1c00 2024-08-18T13:39:59.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9cdd5700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc800053c0 0x7fcc9815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f87594d6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f874c0dc2b0 0x7f874c0e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7ed5d700 1 -- start start 2024-08-18T13:39:59.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f87594d6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f874c0450f0 0x7f874c045570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f87594d6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c04b220 con 0x7f874c04cc40 2024-08-18T13:39:59.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f87594d6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c0e30a0 con 0x7f874c0450f0 2024-08-18T13:39:59.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f87594d6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c04d620 con 0x7f874c0dc2b0 2024-08-18T13:39:59.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9d5d6700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc9814dfc0 0x7fcc9814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316411897 0 0) 0x7f1ef00c72e0 con 0x7f1ef00c1830 2024-08-18T13:39:59.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed8003570 con 0x7f1ef00c1830 2024-08-18T13:39:59.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f874c0dc2b0 0x7f874c0e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f874c0dc2b0 0x7f874c0e2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:43848/0 (socket says 172.21.15.118:43848) 2024-08-18T13:39:59.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 -- 172.21.15.118:0/472886030 learned_addr learned my addr 172.21.15.118:0/472886030 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8758cd5700 1 --2- 172.21.15.118:0/472886030 >> v2:172.21.15.118:3300/0 conn(0x7f874c0450f0 0x7f874c045570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744762017 0 0) 0x7f1ef0155e60 con 0x7f1ef00c23b0 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef00c72e0 con 0x7f1ef00c23b0 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3517031178 0 0) 0x7f1ed8003570 con 0x7f1ef00c1830 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 >> v2:172.21.15.118:3300/0 conn(0x7f1ef00c1c00 msgr2=0x7f1ef00c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 --2- 172.21.15.118:0/4116874991 >> v2:172.21.15.118:3300/0 conn(0x7f1ef00c1c00 0x7f1ef00c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 >> v1:172.21.15.178:6789/0 conn(0x7f1ef00c23b0 legacy=0x7f1ef00c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef01560e0 con 0x7f1ef00c1830 2024-08-18T13:39:59.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f1ee4001470 con 0x7f1ef00c1830 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 >> v1:172.21.15.118:6789/0 conn(0x7f1ef00c1830 legacy=0x7f1ef00673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a3fff700 1 -- start start 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a406f2f0 0x7f93a415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8003bf0 0x7f1ed8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed8004810 0x7f1ed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ed80051c0 0x7f1ed800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed800c130 con 0x7f1ed8004810 2024-08-18T13:39:59.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed800c2a0 con 0x7f1ed8003bf0 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed800c460 con 0x7f1ed80051c0 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a3fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a40c8200 0x7f93a414ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a4153690 0x7f93a414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4155020 con 0x7f93a4153690 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a3fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a415d3e0 con 0x7f93a406f2f0 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a3fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4153d30 con 0x7f93a40c8200 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9cdd5700 1 -- 172.21.15.118:0/1576157988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc9814dfc0 msgr2=0x7fcc9814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9cdd5700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc9814dfc0 0x7fcc9814e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9cdd5700 1 -- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc980c7100 msgr2=0x7fcc980bf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9cdd5700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc980c7100 0x7fcc980bf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9cdd5700 1 -- 172.21.15.118:0/1576157988 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9814e970 con 0x7fcc800053c0 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf77fe700 1 -- 172.21.15.118:0/324346889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf80c8290 msgr2=0x7f8bf80c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf77fe700 1 --2- 172.21.15.118:0/324346889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf80c8290 0x7f8bf80c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf77fe700 1 -- 172.21.15.118:0/324346889 >> v2:172.21.15.118:3300/0 conn(0x7f8bf815c050 msgr2=0x7f8bf80673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf77fe700 1 --2- 172.21.15.118:0/324346889 >> v2:172.21.15.118:3300/0 conn(0x7f8bf815c050 0x7f8bf80673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf77fe700 1 -- 172.21.15.118:0/324346889 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf806e390 con 0x7f8bf8161e70 2024-08-18T13:39:59.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf6ffd700 1 --2- 172.21.15.118:0/324346889 >> v2:172.21.15.118:3300/0 conn(0x7f8bf815c050 0x7f8bf80673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:59.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7ed5d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d780c0b00 0x7f5d78068820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 -- 172.21.15.118:0/472886030 >> v2:172.21.15.118:3300/0 conn(0x7f874c0450f0 msgr2=0x7f874c045570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 --2- 172.21.15.118:0/472886030 >> v2:172.21.15.118:3300/0 conn(0x7f874c0450f0 0x7f874c045570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 -- 172.21.15.118:0/472886030 >> v1:172.21.15.178:6789/0 conn(0x7f874c04cc40 legacy=0x7f874c006da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 -- 172.21.15.118:0/472886030 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c04d380 con 0x7f874c0dc2b0 2024-08-18T13:39:59.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8758cd5700 1 --2- 172.21.15.118:0/472886030 >> v2:172.21.15.118:3300/0 conn(0x7f874c0450f0 0x7f874c045570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:59.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7ed5d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5d780c8d40 0x7f5d780c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7ed5d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780c15e0 con 0x7f5d78155c90 2024-08-18T13:39:59.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7ed5d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780c7bf0 con 0x7f5d780c8d40 2024-08-18T13:39:59.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7ed5d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781561b0 con 0x7f5d780c0b00 2024-08-18T13:39:59.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeeffd700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed8004810 0x7f1ed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a27fc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a40c8200 0x7f93a414ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a4153690 0x7f93a414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a4153690 0x7f93a414ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45096/0 (socket says 172.21.15.118:45096) 2024-08-18T13:39:59.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 -- 172.21.15.118:0/1340300574 learned_addr learned my addr 172.21.15.118:0/1340300574 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9cdd5700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc800053c0 0x7fcc9815e280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d6f0 tx=0x7fcc8c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a2ffd700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a406f2f0 0x7f93a415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8753fff700 1 --2- 172.21.15.118:0/472886030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0dc2b0 0x7f874c0e2b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8754065f30 tx=0x7f8754066810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa121422e98a76c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7dd5b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d780c0b00 0x7f5d78068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7dd5b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d780c0b00 0x7f5d78068820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54276/0 (socket says 172.21.15.118:54276) 2024-08-18T13:39:59.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7dd5b700 1 -- 172.21.15.118:0/3739217982 learned_addr learned my addr 172.21.15.118:0/3739217982 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7e55c700 1 --2- 172.21.15.118:0/3739217982 >> v2:172.21.15.118:3300/0 conn(0x7f5d780c8d40 0x7f5d780c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeffff700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ed80051c0 0x7f1ed800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eef7fe700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8003bf0 0x7f1ed8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeeffd700 1 -- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ed80051c0 msgr2=0x7f1ed800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeeffd700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ed80051c0 0x7f1ed800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeeffd700 1 -- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8003bf0 msgr2=0x7f1ed8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeeffd700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8003bf0 0x7f1ed8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeeffd700 1 -- 172.21.15.118:0/4116874991 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef01560e0 con 0x7f1ed8004810 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7e55c700 1 -- 172.21.15.118:0/3739217982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d780c0b00 msgr2=0x7f5d78068820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7e55c700 1 --2- 172.21.15.118:0/3739217982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d780c0b00 0x7f5d78068820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7e55c700 1 -- 172.21.15.118:0/3739217982 >> v1:172.21.15.178:6789/0 conn(0x7f5d78155c90 legacy=0x7f5d780c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:39:59.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8bf77fe700 1 --2- 172.21.15.118:0/324346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf8161e70 0x7f8bf80c27e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bf806d360 tx=0x7f8be80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1646a3be1635e76c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7e55c700 1 -- 172.21.15.118:0/3739217982 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780c7000 con 0x7f5d780c8d40 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc94ff9700 1 -- 172.21.15.118:0/1576157988 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c012670 con 0x7fcc800053c0 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc94ff9700 1 -- 172.21.15.118:0/1576157988 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00c040 con 0x7fcc800053c0 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc981592b0 con 0x7fcc800053c0 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9814f5a0 con 0x7fcc800053c0 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 -- 172.21.15.118:0/1340300574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a40c8200 msgr2=0x7f93a414ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93a40c8200 0x7f93a414ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 -- 172.21.15.118:0/1340300574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a406f2f0 msgr2=0x7f93a415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a406f2f0 0x7f93a415a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 -- 172.21.15.118:0/1340300574 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c009a20 con 0x7f93a4153690 2024-08-18T13:39:59.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a2ffd700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a406f2f0 0x7f93a415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:39:59.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7e55c700 1 --2- 172.21.15.118:0/3739217982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c8d40 0x7f5d780c8080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009e60 tx=0x7f5d70009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdb223497dbd44f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7cd59700 1 -- 172.21.15.118:0/3739217982 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000e070 con 0x7f5d780c8d40 2024-08-18T13:39:59.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f5d7cd59700 1 -- 172.21.15.118:0/3739217982 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7000bbb0 con 0x7f5d780c8d40 2024-08-18T13:39:59.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeffff700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ed80051c0 0x7f1ed800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:39:59.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeffff700 1 -- 172.21.15.118:0/4116874991 reap_dead start 2024-08-18T13:39:59.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eeeffd700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed8004810 0x7f1ed8004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006f00 tx=0x7f1ee00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e28aa93070beeac server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8752ffd700 1 -- 172.21.15.118:0/472886030 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875404c240 con 0x7f874c0dc2b0 2024-08-18T13:39:59.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8752ffd700 1 -- 172.21.15.118:0/472886030 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87540008d0 con 0x7f874c0dc2b0 2024-08-18T13:39:59.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f8752ffd700 1 -- 172.21.15.118:0/472886030 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754070500 con 0x7f874c0dc2b0 2024-08-18T13:39:59.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7fcc94ff9700 1 -- 172.21.15.118:0/1576157988 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c012a60 con 0x7fcc800053c0 2024-08-18T13:39:59.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000e6e0 con 0x7f1ed8004810 2024-08-18T13:39:59.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000e840 con 0x7f1ed8004810 2024-08-18T13:39:59.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1eee7fc700 1 -- 172.21.15.118:0/4116874991 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000ec00 con 0x7f1ed8004810 2024-08-18T13:39:59.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f8bf5ffb700 1 -- 172.21.15.118:0/324346889 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800b6b0 con 0x7f8bf8161e70 2024-08-18T13:39:59.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 -- 172.21.15.118:0/472886030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0dc2b0 msgr2=0x7f874c0e2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.946+0000 7f93a37fe700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a4153690 0x7f93a414ef40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f939c0129a0 tx=0x7f939c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 --2- 172.21.15.118:0/472886030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0dc2b0 0x7f874c0e2b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8754065f30 tx=0x7f8754066810 comp rx=0 tx=0).stop 2024-08-18T13:39:59.981 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f8bf5ffb700 1 -- 172.21.15.118:0/324346889 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800b810 con 0x7f8bf8161e70 2024-08-18T13:39:59.981 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f8bf5ffb700 1 -- 172.21.15.118:0/324346889 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800bbd0 con 0x7f8bf8161e70 2024-08-18T13:39:59.981 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/3739217982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c8d40 msgr2=0x7f5d780c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.981 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 --2- 172.21.15.118:0/3739217982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c8d40 0x7f5d780c8080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009e60 tx=0x7f5d70009780 comp rx=0 tx=0).stop 2024-08-18T13:39:59.981 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0070c0 con 0x7f93a4153690 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/3739217982 shutdown_connections 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 --2- 172.21.15.118:0/3739217982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c8d40 0x7f5d780c8080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 --2- 172.21.15.118:0/3739217982 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d780c0b00 0x7f5d78068820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/3739217982 >> 172.21.15.118:0/3739217982 conn(0x7f5d780cc630 msgr2=0x7f5d780c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c012d90 con 0x7f93a4153690 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a40c7310 con 0x7f93a4153690 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0266b0 con 0x7f93a4153690 2024-08-18T13:39:59.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a415ad90 con 0x7f93a4153690 2024-08-18T13:39:59.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/3739217982 shutdown_connections 2024-08-18T13:39:59.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f8bf7fff700 1 -- 172.21.15.118:0/324346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf8161e70 msgr2=0x7f8bf80c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/324346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf8161e70 0x7f8bf80c27e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bf806d360 tx=0x7f8be80091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 -- 172.21.15.118:0/472886030 shutdown_connections 2024-08-18T13:39:59.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 --2- 172.21.15.118:0/472886030 >> v2:172.21.15.118:3300/0 conn(0x7f874c0450f0 0x7f874c045570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 --2- 172.21.15.118:0/472886030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0dc2b0 0x7f874c0e2b60 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8754065f30 tx=0x7f8754066810 comp rx=0 tx=0).stop 2024-08-18T13:39:59.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 -- 172.21.15.118:0/472886030 >> 172.21.15.118:0/472886030 conn(0x7f874c00be40 msgr2=0x7f874c00c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/3739217982 wait complete. 2024-08-18T13:39:59.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed8004810 msgr2=0x7f1ed8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1ef5eb6700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed8004810 0x7f1ed8004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006f00 tx=0x7f1ee00091a0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.984 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 shutdown_connections 2024-08-18T13:39:59.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1ef5eb6700 1 --2- 172.21.15.118:0/4116874991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ed8004810 0x7f1ed8004c80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 >> 172.21.15.118:0/4116874991 conn(0x7f1ef00c44b0 msgr2=0x7f1ef00c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 shutdown_connections 2024-08-18T13:39:59.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/4116874991 wait complete. 2024-08-18T13:39:59.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc94ff9700 1 -- 172.21.15.118:0/1576157988 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcc8c0113c0 con 0x7fcc800053c0 2024-08-18T13:39:59.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 -- 172.21.15.118:0/472886030 shutdown_connections 2024-08-18T13:39:59.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 -- 172.21.15.118:0/472886030 wait complete. 2024-08-18T13:39:59.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc94ff9700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcc840608b0 0x7fcc84062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc94ff9700 1 -- 172.21.15.118:0/1576157988 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fcc8c07cda0 con 0x7fcc800053c0 2024-08-18T13:39:59.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc97fff700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcc840608b0 0x7fcc84062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc97fff700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcc840608b0 0x7fcc84062d60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc980bff50 tx=0x7fcc900016f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 Processor -- start 2024-08-18T13:39:59.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc9ddd7700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7fcc780015f0 0x7fcc78003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 --> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc78006b90 con 0x7fcc780015f0 2024-08-18T13:39:59.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93a4071410 con 0x7f93a4153690 2024-08-18T13:39:59.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 Processor -- start 2024-08-18T13:39:59.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 -- start start 2024-08-18T13:39:59.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7fcc9d5d6700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7fcc780015f0 0x7fcc78003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 -- 172.21.15.118:0/324346889 shutdown_connections 2024-08-18T13:39:59.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/324346889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf8161e70 0x7f8bf80c27e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/324346889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf80c8290 0x7f8bf80c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/324346889 >> v2:172.21.15.118:3300/0 conn(0x7f8bf815c050 0x7f8bf80673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7fcc9d5d6700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7fcc780015f0 0x7fcc78003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 -- 172.21.15.118:0/324346889 >> 172.21.15.118:0/324346889 conn(0x7f8bf806c020 msgr2=0x7f8bf80c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:39:59.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 -- 172.21.15.118:0/324346889 shutdown_connections 2024-08-18T13:39:59.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 -- 172.21.15.118:0/324346889 wait complete. 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 Processor -- start 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f87594d6700 1 -- start start 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87594d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0450f0 0x7f874c04f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87594d6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c04cc40 0x7f874c0e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87594d6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f874c0e3300 0x7f874c0e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87594d6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0f65b0 con 0x7f874c0e3300 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87594d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0f5700 con 0x7f874c0450f0 2024-08-18T13:39:59.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87594d6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0f59a0 con 0x7f874c04cc40 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.950+0000 7f5d7ed5d700 1 -- start start 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7ed5d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c0b00 0x7f5d78145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7ed5d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d780c8d40 0x7f5d78068030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7ed5d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d78155c90 0x7f5d7814c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7ed5d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78148b80 con 0x7f5d780c0b00 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7ed5d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781544d0 con 0x7f5d780c8d40 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7ed5d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78148530 con 0x7f5d78155c90 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c0b00 0x7f5d78145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c0b00 0x7f5d78145ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45116/0 (socket says 172.21.15.118:45116) 2024-08-18T13:39:59.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 -- 172.21.15.118:0/2103048114 learned_addr learned my addr 172.21.15.118:0/2103048114 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f874c0e3300 0x7f874c0e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f874c0e3300 0x7f874c0e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45130/0 (socket says 172.21.15.118:45130) 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 -- 172.21.15.118:0/2704180677 learned_addr learned my addr 172.21.15.118:0/2704180677 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7d55a700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d780c8d40 0x7f5d78068030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7e55c700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d78155c90 0x7f5d7814c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 -- 172.21.15.118:0/2103048114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d78155c90 msgr2=0x7f5d7814c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d78155c90 0x7f5d7814c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 -- 172.21.15.118:0/2103048114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d780c8d40 msgr2=0x7f5d78068030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d780c8d40 0x7f5d78068030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 -- 172.21.15.118:0/2103048114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009a20 con 0x7f5d780c0b00 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7e55c700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d78155c90 0x7f5d7814c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8004810 0x7f1ef014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef014bf50 0x7f1ef0168810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef0168d50 0x7f1ef01492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef014a3a0 con 0x7f1ef0168d50 2024-08-18T13:39:59.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eef7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8004810 0x7f1ef014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87537fe700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c04cc40 0x7f874c0e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8753fff700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0450f0 0x7f874c04f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7fcc94ff9700 1 -- 172.21.15.118:0/1576157988 <== osd.3 v2:172.21.15.118:6802/24280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fcc78006b90 con 0x7fcc780015f0 2024-08-18T13:39:59.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eef7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8004810 0x7f1ef014ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:43894/0 (socket says 172.21.15.118:43894) 2024-08-18T13:39:59.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eef7fe700 1 -- 172.21.15.118:0/3649800882 learned_addr learned my addr 172.21.15.118:0/3649800882 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:39:59.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01694e0 con 0x7f1ed8004810 2024-08-18T13:39:59.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef014b020 con 0x7f1ef014bf50 2024-08-18T13:39:59.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7dd5b700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c0b00 0x7f5d78145ad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400ea50 tx=0x7f5d74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f939c01d420 con 0x7f93a4153690 2024-08-18T13:39:59.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeffff700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef0168d50 0x7f1ef01492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 -- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c04cc40 msgr2=0x7f874c0e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f874c04cc40 0x7f874c0e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 -- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0450f0 msgr2=0x7f874c04f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c0450f0 0x7f874c04f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 -- 172.21.15.118:0/2704180677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875404c770 con 0x7f874c0e3300 2024-08-18T13:39:59.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8758cd5700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f874c0e3300 0x7f874c0e4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f874800d860 tx=0x7f8748008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeeffd700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef014bf50 0x7f1ef0168810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeffff700 1 -- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef014bf50 msgr2=0x7f1ef0168810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeffff700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ef014bf50 0x7f1ef0168810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeffff700 1 -- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8004810 msgr2=0x7f1ef014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:39:59.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeffff700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed8004810 0x7f1ef014ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:39:59.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeffff700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef014c3c0 con 0x7f1ef0168d50 2024-08-18T13:39:59.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f938affd700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9390060960 0x7f9390062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f1eeffff700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef0168d50 0x7f1ef01492f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ef006c820 tx=0x7f1ee8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:39:59.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f939c07f820 con 0x7f93a4153690 2024-08-18T13:39:59.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f93a2ffd700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9390060960 0x7f9390062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f938affd700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7f9390069f90 0x7f939006c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:39:59.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 --> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f939006ca90 con 0x7f9390069f90 2024-08-18T13:39:59.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f939c07fbf0 con 0x7f93a4153690 2024-08-18T13:39:59.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f93a27fc700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7f9390069f90 0x7f939006c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:39:59.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 Processor -- start 2024-08-18T13:39:59.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400eee0 con 0x7f5d780c0b00 2024-08-18T13:39:59.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f04759a2700 1 Processor -- start 2024-08-18T13:40:00.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7814beb0 con 0x7f5d780c0b00 2024-08-18T13:40:00.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7400c850 con 0x7f5d780c0b00 2024-08-18T13:40:00.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f93a2ffd700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9390060960 0x7f9390062e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f93940113e0 tx=0x7f93940091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f8bf7fff700 1 -- start start 2024-08-18T13:40:00.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87480125f0 con 0x7f874c0e3300 2024-08-18T13:40:00.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f93a27fc700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7f9390069f90 0x7f939006c3c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8748012750 con 0x7f874c0e3300 2024-08-18T13:40:00.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.954+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c0f5c80 con 0x7f874c0e3300 2024-08-18T13:40:00.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c0f6240 con 0x7f874c0e3300 2024-08-18T13:40:00.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400cbd0 con 0x7f5d780c0b00 2024-08-18T13:40:00.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7814dba0 con 0x7f5d780c0b00 2024-08-18T13:40:00.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80032e0 con 0x7f1ef0168d50 2024-08-18T13:40:00.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee8003440 con 0x7f1ef0168d50 2024-08-18T13:40:00.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8006df0 con 0x7f1ef0168d50 2024-08-18T13:40:00.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== osd.1 v2:172.21.15.118:6816/24279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f939006ca90 con 0x7f9390069f90 2024-08-18T13:40:00.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef0068540 con 0x7f1ef0168d50 2024-08-18T13:40:00.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef014b230 con 0x7f1ef0168d50 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf80c8290 0x7f8bf80c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf815c050 0x7f8bf8176140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bf8161e70 0x7f8bf8176680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf817a200 con 0x7f8bf8161e70 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf814e7b0 con 0x7f8bf80c8290 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8174f50 con 0x7f8bf815c050 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf67fc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf815c050 0x7f8bf8176140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bf8161e70 0x7f8bf8176680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf67fc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf815c050 0x7f8bf8176140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54310/0 (socket says 172.21.15.118:54310) 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf67fc700 1 -- 172.21.15.118:0/1037222276 learned_addr learned my addr 172.21.15.118:0/1037222276 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf6ffd700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf80c8290 0x7f8bf80c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf6ffd700 1 -- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf815c050 msgr2=0x7f8bf8176140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf6ffd700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bf815c050 0x7f8bf8176140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf6ffd700 1 -- 172.21.15.118:0/1037222276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bf8161e70 msgr2=0x7f8bf8176680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf6ffd700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bf8161e70 0x7f8bf8176680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf6ffd700 1 -- 172.21.15.118:0/1037222276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8009420 con 0x7f8bf80c8290 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04759a2700 1 -- start start 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04759a2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f047014b630 0x7f04700673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04759a2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f04700c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04759a2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f047006e940 0x7f047006edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04759a2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04700c8820 con 0x7f047006e940 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04759a2700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04700c2ae0 con 0x7f047014b630 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04759a2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047006f720 con 0x7f04700c8410 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04749a0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f047014b630 0x7f04700673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04749a0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f047014b630 0x7f04700673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:43906/0 (socket says 172.21.15.118:43906) 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04749a0700 1 -- 172.21.15.118:0/2736396130 learned_addr learned my addr 172.21.15.118:0/2736396130 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f046ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f04700c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f1ed57fa700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ef0071480 con 0x7f1ef0168d50 2024-08-18T13:40:00.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf6ffd700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf80c8290 0x7f8bf80c6ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0007eb0 tx=0x7f8bf0009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748012ad0 con 0x7f874c0e3300 2024-08-18T13:40:00.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bdeffd700 1 -- 172.21.15.118:0/1037222276 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf001ec00 con 0x7f8bf80c8290 2024-08-18T13:40:00.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04751a1700 1 --2- 172.21.15.118:0/2736396130 >> v2:172.21.15.118:3300/0 conn(0x7f047006e940 0x7f047006edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf81751f0 con 0x7f8bf80c8290 2024-08-18T13:40:00.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf81798d0 con 0x7f8bf80c8290 2024-08-18T13:40:00.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f046ffff700 1 -- 172.21.15.118:0/2736396130 >> v2:172.21.15.178:3300/0 conn(0x7f047014b630 msgr2=0x7f04700673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f046ffff700 1 --2- 172.21.15.118:0/2736396130 >> v2:172.21.15.178:3300/0 conn(0x7f047014b630 0x7f04700673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f046ffff700 1 -- 172.21.15.118:0/2736396130 >> v2:172.21.15.118:3300/0 conn(0x7f047006e940 msgr2=0x7f047006edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f046ffff700 1 --2- 172.21.15.118:0/2736396130 >> v2:172.21.15.118:3300/0 conn(0x7f047006e940 0x7f047006edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f046ffff700 1 -- 172.21.15.118:0/2736396130 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470155430 con 0x7f04700c8410 2024-08-18T13:40:00.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8734000fd0 con 0x7f874c0e3300 2024-08-18T13:40:00.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f04751a1700 1 --2- 172.21.15.118:0/2736396130 >> v2:172.21.15.118:3300/0 conn(0x7f047006e940 0x7f047006edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:40:00.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8748013480 con 0x7f874c0e3300 2024-08-18T13:40:00.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f046ffff700 1 --2- 172.21.15.118:0/2736396130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f04700c2f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0470069870 tx=0x7f046400d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95bc05e27f5f001b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bdeffd700 1 -- 172.21.15.118:0/1037222276 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf001ed60 con 0x7f8bf80c8290 2024-08-18T13:40:00.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f8bdeffd700 1 -- 172.21.15.118:0/1037222276 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0026630 con 0x7f8bf80c8290 2024-08-18T13:40:00.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f046f7fe700 1 -- 172.21.15.118:0/2736396130 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464012070 con 0x7f04700c8410 2024-08-18T13:40:00.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f046f7fe700 1 -- 172.21.15.118:0/2736396130 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046400d2e0 con 0x7f04700c8410 2024-08-18T13:40:00.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f046f7fe700 1 -- 172.21.15.118:0/2736396130 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046400f9b0 con 0x7f04700c8410 2024-08-18T13:40:00.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1ee80018f0 con 0x7f1ef0168d50 2024-08-18T13:40:00.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f873ffff700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8744060a30 0x7f8744062ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f874807cea0 con 0x7f874c0e3300 2024-08-18T13:40:00.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f873ffff700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7f874406a060 0x7f874406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 --> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f874406cb60 con 0x7f874406a060 2024-08-18T13:40:00.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f874807f4e0 con 0x7f874c0e3300 2024-08-18T13:40:00.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f87537fe700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7f874406a060 0x7f874406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bdeffd700 1 -- 172.21.15.118:0/1037222276 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8bf0026a20 con 0x7f8bf80c8290 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 -- 172.21.15.118:0/2736396130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 msgr2=0x7f04700c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 --2- 172.21.15.118:0/2736396130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f04700c2f70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0470069870 tx=0x7f046400d630 comp rx=0 tx=0).stop 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5d74013470 con 0x7f5d780c0b00 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d78071410 con 0x7f5d780c0b00 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d6dffb700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5d5c0608e0 0x7f5d5c062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d7d55a700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5d5c0608e0 0x7f5d5c062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f5d74051590 con 0x7f5d780c0b00 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d6dffb700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f5d5c069d20 0x7f5d5c06c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 --> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d5c06c820 con 0x7f5d5c069d20 2024-08-18T13:40:00.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d7e55c700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f5d5c069d20 0x7f5d5c06c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.958+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d7404d190 con 0x7f5d780c0b00 2024-08-18T13:40:00.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f5d7e55c700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f5d5c069d20 0x7f5d5c06c150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f5d7d55a700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5d5c0608e0 0x7f5d5c062d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5d7814e4a0 tx=0x7f5d68009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f87537fe700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7f874406a060 0x7f874406c490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== osd.0 v2:172.21.15.118:6824/24281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5d5c06c820 con 0x7f5d5c069d20 2024-08-18T13:40:00.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1ed77fe700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1edc060960 0x7f1edc062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bdeffd700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8be4060930 0x7f8be4062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bdeffd700 1 -- 172.21.15.118:0/1037222276 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f8bf008e050 con 0x7f8bf80c8290 2024-08-18T13:40:00.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f1ee807c5e0 con 0x7f1ef0168d50 2024-08-18T13:40:00.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bf67fc700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8be4060930 0x7f8be4062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1ed77fe700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f1edc069f90 0x7f1edc06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1edc06ca90 con 0x7f1edc069f90 2024-08-18T13:40:00.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee807c9b0 con 0x7f1ef0168d50 2024-08-18T13:40:00.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1eeeffd700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f1edc069f90 0x7f1edc06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bf67fc700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8be4060930 0x7f8be4062de0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80c6360 tx=0x7f8bec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8753fff700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8744060a30 0x7f8744062ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 -- 172.21.15.118:0/2736396130 shutdown_connections 2024-08-18T13:40:00.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 --2- 172.21.15.118:0/2736396130 >> v2:172.21.15.118:3300/0 conn(0x7f047006e940 0x7f047006edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 --2- 172.21.15.118:0/2736396130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f04700c2f70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f8bd80015f0 0x7f8bd8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 --> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8bd8006b90 con 0x7f8bd80015f0 2024-08-18T13:40:00.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== osd.7 v2:172.21.15.178:6800/24440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f874406cb60 con 0x7f874406a060 2024-08-18T13:40:00.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f1eeeffd700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f1edc069f90 0x7f1edc06c3c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 --2- 172.21.15.118:0/2736396130 >> v2:172.21.15.178:3300/0 conn(0x7f047014b630 0x7f04700673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 -- 172.21.15.118:0/2736396130 >> 172.21.15.118:0/2736396130 conn(0x7f047006c020 msgr2=0x7f04700c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 -- 172.21.15.118:0/2736396130 shutdown_connections 2024-08-18T13:40:00.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f04759a2700 1 -- 172.21.15.118:0/2736396130 wait complete. 2024-08-18T13:40:00.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8bf77fe700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f8bd80015f0 0x7f8bd8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f8753fff700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8744060a30 0x7f8744062ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f874c046fa0 tx=0x7f875406c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c996a700 1 Processor -- start 2024-08-18T13:40:00.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c996a700 1 -- start start 2024-08-18T13:40:00.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c996a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f74c40673d0 0x7f74c406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c996a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c406edf0 con 0x7f74c4153690 2024-08-18T13:40:00.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f1eef7fe700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1edc060960 0x7f1edc062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c996a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c415a0b0 con 0x7f74c40c8200 2024-08-18T13:40:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f8bf77fe700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f8bd80015f0 0x7f8bd8003aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f1eef7fe700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1edc060960 0x7f1edc062e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4001ed0 tx=0x7f1ee4007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 Processor -- start 2024-08-18T13:40:00.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 -- start start 2024-08-18T13:40:00.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== osd.4 v2:172.21.15.178:6808/24441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1edc06ca90 con 0x7f1edc069f90 2024-08-18T13:40:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f047006e940 0x7f0470149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f0470151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f047014b630 0x7f04700c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04701501b0 con 0x7f047006e940 2024-08-18T13:40:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04700c0a60 con 0x7f047014b630 2024-08-18T13:40:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04759a2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0470161220 con 0x7f04700c8410 2024-08-18T13:40:00.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f047006e940 0x7f0470149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f047006e940 0x7f0470149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45180/0 (socket says 172.21.15.118:45180) 2024-08-18T13:40:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 -- 172.21.15.118:0/2041572740 learned_addr learned my addr 172.21.15.118:0/2041572740 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f046ffff700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f0470151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04751a1700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f047014b630 0x7f04700c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f8bdeffd700 1 -- 172.21.15.118:0/1037222276 <== osd.5 v2:172.21.15.178:6817/24442 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8bd8006b90 con 0x7f8bd80015f0 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c996a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c415a5f0 con 0x7f74c40673d0 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c8968700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f74c4153690 0x7f74c40c22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45908/0 (socket says 172.21.15.118:45908) 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c8968700 1 -- 172.21.15.118:0/2138549084 learned_addr learned my addr 172.21.15.118:0/2138549084 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.962+0000 7f74c3fff700 1 --2- 172.21.15.118:0/2138549084 >> v2:172.21.15.178:3300/0 conn(0x7f74c40673d0 0x7f74c406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c3fff700 1 -- 172.21.15.118:0/2138549084 >> v1:172.21.15.178:6789/0 conn(0x7f74c40c8200 legacy=0x7f74c40c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c3fff700 1 -- 172.21.15.118:0/2138549084 >> v1:172.21.15.118:6789/0 conn(0x7f74c4153690 legacy=0x7f74c40c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c3fff700 1 -- 172.21.15.118:0/2138549084 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4153d10 con 0x7f74c40673d0 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c3fff700 1 --2- 172.21.15.118:0/2138549084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 0x7f74c406f280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009e60 tx=0x7f74bc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adda2012df79be03 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c37fe700 1 -- 172.21.15.118:0/2138549084 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00e070 con 0x7f74c40673d0 2024-08-18T13:40:00.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c37fe700 1 -- 172.21.15.118:0/2138549084 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc00bb30 con 0x7f74c40673d0 2024-08-18T13:40:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c37fe700 1 -- 172.21.15.118:0/2138549084 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00bed0 con 0x7f74c40673d0 2024-08-18T13:40:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c996a700 1 -- 172.21.15.118:0/2138549084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 msgr2=0x7f74c406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c996a700 1 --2- 172.21.15.118:0/2138549084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 0x7f74c406f280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009e60 tx=0x7f74bc009780 comp rx=0 tx=0).stop 2024-08-18T13:40:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 -- 172.21.15.118:0/2041572740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 msgr2=0x7f0470151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c996a700 1 -- 172.21.15.118:0/2138549084 shutdown_connections 2024-08-18T13:40:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c996a700 1 --2- 172.21.15.118:0/2138549084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 0x7f74c406f280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c996a700 1 -- 172.21.15.118:0/2138549084 >> 172.21.15.118:0/2138549084 conn(0x7f74c406c020 msgr2=0x7f74c40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f04700c8410 0x7f0470151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 -- 172.21.15.118:0/2041572740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f047014b630 msgr2=0x7f04700c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f047014b630 0x7f04700c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 -- 172.21.15.118:0/2041572740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046400d8d0 con 0x7f047006e940 2024-08-18T13:40:00.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c996a700 1 -- 172.21.15.118:0/2138549084 shutdown_connections 2024-08-18T13:40:00.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f04749a0700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f047006e940 0x7f0470149e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f046000baf0 tx=0x7f046000d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.966+0000 7f74c996a700 1 -- 172.21.15.118:0/2138549084 wait complete. 2024-08-18T13:40:00.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 Processor -- start 2024-08-18T13:40:00.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 -- start start 2024-08-18T13:40:00.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 0x7f74c40befb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c40c8200 0x7f74c40c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c4153690 0x7f74c40c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4142e60 con 0x7f74c4153690 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4141af0 con 0x7f74c40673d0 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c996a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4141810 con 0x7f74c40c8200 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c4153690 0x7f74c40c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c4153690 0x7f74c40c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45188/0 (socket says 172.21.15.118:45188) 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 -- 172.21.15.118:0/2913009875 learned_addr learned my addr 172.21.15.118:0/2913009875 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c8968700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 0x7f74c40befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c3fff700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c40c8200 0x7f74c40c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 -- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c40c8200 msgr2=0x7f74c40c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74c40c8200 0x7f74c40c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 -- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 msgr2=0x7f74c40befb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74c40673d0 0x7f74c40befb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 -- 172.21.15.118:0/2913009875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009a20 con 0x7f74c4153690 2024-08-18T13:40:00.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.970+0000 7f74c9169700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c4153690 0x7f74c40c4a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c590 tx=0x7f74b8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.974+0000 7f046cff9700 1 -- 172.21.15.118:0/2041572740 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046000eb90 con 0x7f047006e940 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.974+0000 7f046cff9700 1 -- 172.21.15.118:0/2041572740 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046000ecf0 con 0x7f047006e940 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.974+0000 7f046cff9700 1 -- 172.21.15.118:0/2041572740 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460016680 con 0x7f047006e940 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.974+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0470161500 con 0x7f047006e940 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.974+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0470161ac0 con 0x7f047006e940 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.974+0000 7f046cff9700 1 -- 172.21.15.118:0/2041572740 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0460016a30 con 0x7f047006e940 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.978+0000 7f046cff9700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0454060930 0x7f0454062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.978+0000 7f046cff9700 1 -- 172.21.15.118:0/2041572740 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f0460010040 con 0x7f047006e940 2024-08-18T13:40:00.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.978+0000 7f046ffff700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0454060930 0x7f0454062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.978+0000 7f046ffff700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0454060930 0x7f0454062de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f046400d8a0 tx=0x7f046401c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f04759a2700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f04701521d0 0x7f0470146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 --> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f047006cf00 con 0x7f04701521d0 2024-08-18T13:40:00.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 --> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc78005c20 con 0x7fcc780015f0 2024-08-18T13:40:00.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 --> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d780c6b10 con 0x7f5d5c069d20 2024-08-18T13:40:00.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f04751a1700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f04701521d0 0x7f0470146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d6dffb700 1 -- 172.21.15.118:0/2103048114 <== osd.0 v2:172.21.15.118:6824/24281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5d780c6b10 con 0x7f5d5c069d20 2024-08-18T13:40:00.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7fcc94ff9700 1 -- 172.21.15.118:0/1576157988 <== osd.3 v2:172.21.15.118:6802/24280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc78005c20 con 0x7fcc780015f0 2024-08-18T13:40:00.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f04751a1700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f04701521d0 0x7f0470146100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f5d5c069d20 msgr2=0x7f5d5c06c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f5d5c069d20 0x7f5d5c06c150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5d5c0608e0 msgr2=0x7f5d5c062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5d5c0608e0 0x7f5d5c062d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5d7814e4a0 tx=0x7f5d68009060 comp rx=0 tx=0).stop 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c0b00 msgr2=0x7f5d78145ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 --2- 172.21.15.118:0/2103048114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d780c0b00 0x7f5d78145ad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400ea50 tx=0x7f5d74009250 comp rx=0 tx=0).stop 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7e55c700 1 -- 172.21.15.118:0/2103048114 reap_dead start 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 shutdown_connections 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 >> 172.21.15.118:0/2103048114 conn(0x7f5d780cc630 msgr2=0x7f5d7814b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 shutdown_connections 2024-08-18T13:40:00.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f046cff9700 1 -- 172.21.15.118:0/2041572740 <== osd.2 v2:172.21.15.118:6800/24282 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f047006cf00 con 0x7f04701521d0 2024-08-18T13:40:00.036 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f5d7ed5d700 1 -- 172.21.15.118:0/2103048114 wait complete. 2024-08-18T13:40:00.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f74c0ff9700 1 -- 172.21.15.118:0/2913009875 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8011590 con 0x7f74c4153690 2024-08-18T13:40:00.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f74c0ff9700 1 -- 172.21.15.118:0/2913009875 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b80116f0 con 0x7f74c4153690 2024-08-18T13:40:00.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c4142530 con 0x7f74c4153690 2024-08-18T13:40:00.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f74c0ff9700 1 -- 172.21.15.118:0/2913009875 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8011ab0 con 0x7f74c4153690 2024-08-18T13:40:00.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.982+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c4142af0 con 0x7f74c4153690 2024-08-18T13:40:00.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.986+0000 7f74c0ff9700 1 -- 172.21.15.118:0/2913009875 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f74b801b3b0 con 0x7f74c4153690 2024-08-18T13:40:00.038 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.986+0000 7f74c0ff9700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74b0060960 0x7f74b0062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.038 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.986+0000 7f74c0ff9700 1 -- 172.21.15.118:0/2913009875 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f74b807c3d0 con 0x7f74c4153690 2024-08-18T13:40:00.038 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.986+0000 7f74c8968700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74b0060960 0x7f74b0062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.038 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.986+0000 7f74c8968700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74b0060960 0x7f74b0062e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f74b40058d0 tx=0x7f74b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.038 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 --> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8bd8005c20 con 0x7f8bd80015f0 2024-08-18T13:40:00.038 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 --> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8734002d30 con 0x7f874406a060 2024-08-18T13:40:00.039 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7fcc780015f0 msgr2=0x7fcc78003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.039 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7fcc780015f0 0x7fcc78003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.039 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f74c996a700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f74a40015f0 0x7f74a4003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.039 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 --> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93a40c7fc0 con 0x7f9390069f90 2024-08-18T13:40:00.040 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 --> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74a4006b90 con 0x7f74a40015f0 2024-08-18T13:40:00.040 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcc840608b0 msgr2=0x7fcc84062d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.040 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcc840608b0 0x7fcc84062d60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc980bff50 tx=0x7fcc900016f0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.040 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc800053c0 msgr2=0x7fcc9815e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.040 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 --2- 172.21.15.118:0/1576157988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc800053c0 0x7fcc9815e280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d6f0 tx=0x7fcc8c008040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.040 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f938affd700 1 -- 172.21.15.118:0/1340300574 <== osd.1 v2:172.21.15.118:6816/24279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f93a40c7fc0 con 0x7f9390069f90 2024-08-18T13:40:00.041 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9d5d6700 1 -- 172.21.15.118:0/1576157988 reap_dead start 2024-08-18T13:40:00.041 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 shutdown_connections 2024-08-18T13:40:00.041 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 >> 172.21.15.118:0/1576157988 conn(0x7fcc9806c020 msgr2=0x7fcc9815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.041 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f74c3fff700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f74a40015f0 0x7f74a4003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bdeffd700 1 -- 172.21.15.118:0/1037222276 <== osd.5 v2:172.21.15.178:6817/24442 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8bd8005c20 con 0x7f8bd80015f0 2024-08-18T13:40:00.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f873ffff700 1 -- 172.21.15.118:0/2704180677 <== osd.7 v2:172.21.15.178:6800/24440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8734002d30 con 0x7f874406a060 2024-08-18T13:40:00.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 shutdown_connections 2024-08-18T13:40:00.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7fcc9ddd7700 1 -- 172.21.15.118:0/1576157988 wait complete. 2024-08-18T13:40:00.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f74c3fff700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f74a40015f0 0x7f74a4003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7f874406a060 msgr2=0x7f874406c490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7f874406a060 0x7f874406c490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f8bd80015f0 msgr2=0x7f8bd8003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8744060a30 msgr2=0x7f8744062ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8744060a30 0x7f8744062ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f874c046fa0 tx=0x7f875406c000 comp rx=0 tx=0).stop 2024-08-18T13:40:00.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f874c0e3300 msgr2=0x7f874c0e4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 --2- 172.21.15.118:0/2704180677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f874c0e3300 0x7f874c0e4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f874800d860 tx=0x7f8748008040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f8bd80015f0 0x7f8bd8003aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8758cd5700 1 -- 172.21.15.118:0/2704180677 reap_dead start 2024-08-18T13:40:00.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8be4060930 msgr2=0x7f8be4062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8be4060930 0x7f8be4062de0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80c6360 tx=0x7f8bec008040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf80c8290 msgr2=0x7f8bf80c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 --2- 172.21.15.118:0/1037222276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bf80c8290 0x7f8bf80c6ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0007eb0 tx=0x7f8bf0009220 comp rx=0 tx=0).stop 2024-08-18T13:40:00.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 shutdown_connections 2024-08-18T13:40:00.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 >> 172.21.15.118:0/2704180677 conn(0x7f874c00be40 msgr2=0x7f874c00c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf77fe700 1 -- 172.21.15.118:0/1037222276 reap_dead start 2024-08-18T13:40:00.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 shutdown_connections 2024-08-18T13:40:00.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 >> 172.21.15.118:0/1037222276 conn(0x7f8bf806c020 msgr2=0x7f8bf81623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 shutdown_connections 2024-08-18T13:40:00.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 shutdown_connections 2024-08-18T13:40:00.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f8bf7fff700 1 -- 172.21.15.118:0/1037222276 wait complete. 2024-08-18T13:40:00.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f87594d6700 1 -- 172.21.15.118:0/2704180677 wait complete. 2024-08-18T13:40:00.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f1ed57fa700 1 -- 172.21.15.118:0/3649800882 --> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ef00689c0 con 0x7f1edc069f90 2024-08-18T13:40:00.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7f9390069f90 msgr2=0x7f939006c3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a3fff700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7f9390069f90 0x7f939006c3c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9390060960 msgr2=0x7f9390062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a3fff700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9390060960 0x7f9390062e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f93940113e0 tx=0x7f93940091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a4153690 msgr2=0x7f93a414ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a3fff700 1 --2- 172.21.15.118:0/1340300574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a4153690 0x7f93a414ef40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f939c0129a0 tx=0x7f939c018000 comp rx=0 tx=0).stop 2024-08-18T13:40:00.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f93a37fe700 1 -- 172.21.15.118:0/1340300574 reap_dead start 2024-08-18T13:40:00.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1ed77fe700 1 -- 172.21.15.118:0/3649800882 <== osd.4 v2:172.21.15.178:6808/24441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1ef00689c0 con 0x7f1edc069f90 2024-08-18T13:40:00.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f1edc069f90 msgr2=0x7f1edc06c3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1ef5eb6700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f1edc069f90 0x7f1edc06c3c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1edc060960 msgr2=0x7f1edc062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1ef5eb6700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1edc060960 0x7f1edc062e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4001ed0 tx=0x7f1ee4007040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef0168d50 msgr2=0x7f1ef01492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1ef5eb6700 1 --2- 172.21.15.118:0/3649800882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1ef0168d50 0x7f1ef01492f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ef006c820 tx=0x7f1ee8009040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f1eeffff700 1 -- 172.21.15.118:0/3649800882 reap_dead start 2024-08-18T13:40:00.050 INFO:teuthology.orchestra.run.smithi118.stdout:60129542154 2024-08-18T13:40:00.051 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-18T13:40:00.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.990+0000 7f74c0ff9700 1 -- 172.21.15.118:0/2913009875 <== osd.6 v2:172.21.15.178:6816/24448 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f74a4006b90 con 0x7f74a40015f0 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.998+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 shutdown_connections 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.998+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 >> 172.21.15.118:0/3649800882 conn(0x7f1ef00c44b0 msgr2=0x7f1ef00c27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.998+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 shutdown_connections 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.998+0000 7f1ef5eb6700 1 -- 172.21.15.118:0/3649800882 wait complete. 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.994+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 shutdown_connections 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.998+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 >> 172.21.15.118:0/1340300574 conn(0x7f93a406c020 msgr2=0x7f93a406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.998+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 shutdown_connections 2024-08-18T13:40:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:39:59.998+0000 7f93a3fff700 1 -- 172.21.15.118:0/1340300574 wait complete. 2024-08-18T13:40:00.053 INFO:teuthology.orchestra.run.smithi118.stdout:64424509451 2024-08-18T13:40:00.053 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-18T13:40:00.054 INFO:teuthology.orchestra.run.smithi118.stdout:60129542155 2024-08-18T13:40:00.054 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-18T13:40:00.055 INFO:teuthology.orchestra.run.smithi118.stdout:55834574857 2024-08-18T13:40:00.055 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-18T13:40:00.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 --> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f74a4005c20 con 0x7f74a40015f0 2024-08-18T13:40:00.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c0ff9700 1 -- 172.21.15.118:0/2913009875 <== osd.6 v2:172.21.15.178:6816/24448 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f74a4005c20 con 0x7f74a40015f0 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f74a40015f0 msgr2=0x7f74a4003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f74a40015f0 0x7f74a4003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74b0060960 msgr2=0x7f74b0062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f74b0060960 0x7f74b0062e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f74b40058d0 tx=0x7f74b4009040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c4153690 msgr2=0x7f74c40c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 --2- 172.21.15.118:0/2913009875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74c4153690 0x7f74c40c4a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f74b800c590 tx=0x7f74b8007ef0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c9169700 1 -- 172.21.15.118:0/2913009875 reap_dead start 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 shutdown_connections 2024-08-18T13:40:00.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 >> 172.21.15.118:0/2913009875 conn(0x7f74c406c020 msgr2=0x7f74c4068140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.058 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 shutdown_connections 2024-08-18T13:40:00.058 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.014+0000 7f74c996a700 1 -- 172.21.15.118:0/2913009875 wait complete. 2024-08-18T13:40:00.058 INFO:teuthology.orchestra.run.smithi118.stdout:60129542154 2024-08-18T13:40:00.058 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-18T13:40:00.059 INFO:teuthology.orchestra.run.smithi118.stdout:64424509450 2024-08-18T13:40:00.060 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-18T13:40:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 --> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04700c1cf0 con 0x7f04701521d0 2024-08-18T13:40:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f046cff9700 1 -- 172.21.15.118:0/2041572740 <== osd.2 v2:172.21.15.118:6800/24282 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04700c1cf0 con 0x7f04701521d0 2024-08-18T13:40:00.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f04701521d0 msgr2=0x7f0470146100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f04701521d0 0x7f0470146100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0454060930 msgr2=0x7f0454062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0454060930 0x7f0454062de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f046400d8a0 tx=0x7f046401c000 comp rx=0 tx=0).stop 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f047006e940 msgr2=0x7f0470149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 --2- 172.21.15.118:0/2041572740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f047006e940 0x7f0470149e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f046000baf0 tx=0x7f046000d040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04751a1700 1 -- 172.21.15.118:0/2041572740 reap_dead start 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 shutdown_connections 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 >> 172.21.15.118:0/2041572740 conn(0x7f047006c020 msgr2=0x7f04700673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 shutdown_connections 2024-08-18T13:40:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.018+0000 7f04759a2700 1 -- 172.21.15.118:0/2041572740 wait complete. 2024-08-18T13:40:00.063 INFO:teuthology.orchestra.run.smithi118.stdout:64424509452 2024-08-18T13:40:00.063 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-18T13:40:00.064 INFO:teuthology.orchestra.run.smithi118.stdout:60129542154 2024-08-18T13:40:00.065 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-18T13:40:00.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f5951968700 1 Processor -- start 2024-08-18T13:40:00.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f5951968700 1 -- start start 2024-08-18T13:40:00.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa612330700 1 Processor -- start 2024-08-18T13:40:00.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa612330700 1 -- start start 2024-08-18T13:40:00.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe914d27700 1 Processor -- start 2024-08-18T13:40:00.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe914d27700 1 -- start start 2024-08-18T13:40:00.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f5951968700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f594c153830 0x7f594c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f5951968700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f594c06f3b0 0x7f594c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f5951968700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594c06e810 con 0x7f594c153c00 2024-08-18T13:40:00.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f5951968700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594c159820 con 0x7f594c06f3b0 2024-08-18T13:40:00.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f5951968700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594c159a30 con 0x7f594c153830 2024-08-18T13:40:00.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fce12fd4700 1 Processor -- start 2024-08-18T13:40:00.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fce12fd4700 1 -- start start 2024-08-18T13:40:00.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa612330700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6040d7d90 0x7fa604036370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa612330700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa60404dce0 0x7fa604046290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f594affd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f594c153830 0x7f594c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f594affd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f594c153830 0x7f594c06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:43938/0 (socket says 172.21.15.118:43938) 2024-08-18T13:40:00.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f594affd700 1 -- 172.21.15.118:0/1508380555 learned_addr learned my addr 172.21.15.118:0/1508380555 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe914d27700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe9100c1b80 0x7fe91006e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa61132e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6040d7d90 0x7fa604036370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa61132e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6040d7d90 0x7fa604036370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45208/0 (socket says 172.21.15.118:45208) 2024-08-18T13:40:00.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa61132e700 1 -- 172.21.15.118:0/1350837693 learned_addr learned my addr 172.21.15.118:0/1350837693 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa611b2f700 1 --1- 172.21.15.118:0/1350837693 >> v1:172.21.15.178:6789/0 conn(0x7fa6040d8160 0x7fa6040467d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:33998/0 (socket says 172.21.15.118:33998) 2024-08-18T13:40:00.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7f594b7fe700 1 --2- 172.21.15.118:0/1508380555 >> v2:172.21.15.118:3300/0 conn(0x7f594c06f3b0 0x7f594c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe914d27700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91006dba0 con 0x7fe9100c11a0 2024-08-18T13:40:00.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe914d27700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9101559f0 con 0x7fe910068b40 2024-08-18T13:40:00.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe914d27700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91006f870 con 0x7fe9100c1b80 2024-08-18T13:40:00.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fce12fd4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c14b5d0 0x7fce0c14b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fce12fd4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce0c155510 0x7fce0c06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fce12fd4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c06a310 con 0x7fce0c06db90 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fce12fd4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c06a800 con 0x7fce0c155510 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fce12fd4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0c3120 con 0x7fce0c14b5d0 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe90ed9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe910068b40 0x7fe9100673d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:34002/0 (socket says 172.21.15.118:34002) 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe90ed9d700 1 -- 172.21.15.118:0/3464231294 learned_addr learned my addr 172.21.15.118:0/3464231294 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fe90dd9b700 1 --2- 172.21.15.118:0/3464231294 >> v2:172.21.15.118:3300/0 conn(0x7fe9100c1b80 0x7fe91006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.238+0000 7fa612330700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60404cb70 con 0x7fa6040d8160 2024-08-18T13:40:00.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 -- 172.21.15.118:0/1350837693 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa604047260 con 0x7fa6040d7d90 2024-08-18T13:40:00.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 -- 172.21.15.118:0/1350837693 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60404c8b0 con 0x7fa60404dce0 2024-08-18T13:40:00.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f594b7fe700 1 -- 172.21.15.118:0/1508380555 >> v2:172.21.15.178:3300/0 conn(0x7f594c153830 msgr2=0x7f594c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f594b7fe700 1 --2- 172.21.15.118:0/1508380555 >> v2:172.21.15.178:3300/0 conn(0x7f594c153830 0x7f594c06d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f594b7fe700 1 -- 172.21.15.118:0/1508380555 >> v1:172.21.15.118:6789/0 conn(0x7f594c153c00 legacy=0x7f594c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f594b7fe700 1 -- 172.21.15.118:0/1508380555 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c159d10 con 0x7f594c06f3b0 2024-08-18T13:40:00.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce0c155510 0x7fce0c06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce0c155510 0x7fce0c06ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:43958/0 (socket says 172.21.15.118:43958) 2024-08-18T13:40:00.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 -- 172.21.15.118:0/3959293096 learned_addr learned my addr 172.21.15.118:0/3959293096 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa610b2d700 1 --2- 172.21.15.118:0/1350837693 >> v2:172.21.15.178:3300/0 conn(0x7fa60404dce0 0x7fa604046290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90d59a700 1 -- 172.21.15.118:0/3464231294 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040746667 0 0) 0x7fe91006dba0 con 0x7fe9100c11a0 2024-08-18T13:40:00.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90d59a700 1 -- 172.21.15.118:0/3464231294 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f8003570 con 0x7fe9100c11a0 2024-08-18T13:40:00.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce10d70700 1 --2- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c14b5d0 0x7fce0c14b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f594b7fe700 1 --2- 172.21.15.118:0/1508380555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c06f3b0 0x7f594c06df20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f594c05ee60 tx=0x7f593c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498420795cd0e1a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa61132e700 1 -- 172.21.15.118:0/1350837693 >> v2:172.21.15.178:3300/0 conn(0x7fa60404dce0 msgr2=0x7fa604046290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa61132e700 1 --2- 172.21.15.118:0/1350837693 >> v2:172.21.15.178:3300/0 conn(0x7fa60404dce0 0x7fa604046290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa61132e700 1 -- 172.21.15.118:0/1350837693 >> v1:172.21.15.178:6789/0 conn(0x7fa6040d8160 legacy=0x7fa6040467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa61132e700 1 -- 172.21.15.118:0/1350837693 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604046ee0 con 0x7fa6040d7d90 2024-08-18T13:40:00.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 -- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c14b5d0 msgr2=0x7fce0c14b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 --2- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c14b5d0 0x7fce0c14b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 -- 172.21.15.118:0/3959293096 >> v1:172.21.15.118:6789/0 conn(0x7fce0c06db90 legacy=0x7fce0c0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90dd9b700 1 -- 172.21.15.118:0/3464231294 >> v1:172.21.15.178:6789/0 conn(0x7fe910068b40 legacy=0x7fe9100673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90dd9b700 1 -- 172.21.15.118:0/3464231294 >> v1:172.21.15.118:6789/0 conn(0x7fe9100c11a0 legacy=0x7fe910067c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90dd9b700 1 -- 172.21.15.118:0/3464231294 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910155cd0 con 0x7fe9100c1b80 2024-08-18T13:40:00.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 -- 172.21.15.118:0/3959293096 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c06a540 con 0x7fce0c155510 2024-08-18T13:40:00.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5949ffb700 1 -- 172.21.15.118:0/1508380555 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c011630 con 0x7f594c06f3b0 2024-08-18T13:40:00.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5949ffb700 1 -- 172.21.15.118:0/1508380555 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c011790 con 0x7f594c06f3b0 2024-08-18T13:40:00.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90dd9b700 1 --2- 172.21.15.118:0/3464231294 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c1b80 0x7fe91006e030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe908009e60 tx=0x7fe908009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26f63e8d45684b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa61132e700 1 --2- 172.21.15.118:0/1350837693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa604036370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa60c065f30 tx=0x7fa60c066810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c9fe9e93b066ff server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce11571700 1 --2- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c155510 0x7fce0c06ec60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce04009f20 tx=0x7fce04009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221e55476268a1e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20793f8700 1 Processor -- start 2024-08-18T13:40:00.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa603fff700 1 -- 172.21.15.118:0/1350837693 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c064cd0 con 0x7fa6040d7d90 2024-08-18T13:40:00.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa603fff700 1 -- 172.21.15.118:0/1350837693 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c0008d0 con 0x7fa6040d7d90 2024-08-18T13:40:00.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa603fff700 1 -- 172.21.15.118:0/1350837693 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c070500 con 0x7fa6040d7d90 2024-08-18T13:40:00.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20793f8700 1 -- start start 2024-08-18T13:40:00.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5949ffb700 1 -- 172.21.15.118:0/1508380555 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c011b30 con 0x7f594c06f3b0 2024-08-18T13:40:00.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 -- 172.21.15.118:0/1508380555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c06f3b0 msgr2=0x7f594c06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 --2- 172.21.15.118:0/1508380555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c06f3b0 0x7f594c06df20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f594c05ee60 tx=0x7f593c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90d59a700 1 -- 172.21.15.118:0/3464231294 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800e070 con 0x7fe9100c1b80 2024-08-18T13:40:00.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90d59a700 1 -- 172.21.15.118:0/3464231294 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90800bb60 con 0x7fe9100c1b80 2024-08-18T13:40:00.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe90d59a700 1 -- 172.21.15.118:0/3464231294 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe908012430 con 0x7fe9100c1b80 2024-08-18T13:40:00.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20793f8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c3a70 0x7f207414cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20793f8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f20740c4fc0 0x7f207414d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20793f8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20740c1be0 con 0x7f20740c3e40 2024-08-18T13:40:00.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20793f8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2074067460 con 0x7f20740c4fc0 2024-08-18T13:40:00.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20793f8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20741531d0 con 0x7f20740c3a70 2024-08-18T13:40:00.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce0b7fe700 1 -- 172.21.15.118:0/3959293096 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400e070 con 0x7fce0c155510 2024-08-18T13:40:00.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 -- 172.21.15.118:0/1350837693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 msgr2=0x7fa604036370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 --2- 172.21.15.118:0/1350837693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa604036370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa60c065f30 tx=0x7fa60c066810 comp rx=0 tx=0).stop 2024-08-18T13:40:00.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce0b7fe700 1 -- 172.21.15.118:0/3959293096 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0400ba40 con 0x7fce0c155510 2024-08-18T13:40:00.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce0b7fe700 1 -- 172.21.15.118:0/3959293096 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040092e0 con 0x7fce0c155510 2024-08-18T13:40:00.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 -- 172.21.15.118:0/1350837693 shutdown_connections 2024-08-18T13:40:00.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 --2- 172.21.15.118:0/1350837693 >> v2:172.21.15.178:3300/0 conn(0x7fa60404dce0 0x7fa604046290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 --2- 172.21.15.118:0/1350837693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa604036370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 -- 172.21.15.118:0/1350837693 >> 172.21.15.118:0/1350837693 conn(0x7fa60400c6d0 msgr2=0x7fa60400caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 -- 172.21.15.118:0/1350837693 shutdown_connections 2024-08-18T13:40:00.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 -- 172.21.15.118:0/1508380555 shutdown_connections 2024-08-18T13:40:00.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 --2- 172.21.15.118:0/1508380555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c06f3b0 0x7f594c06df20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20737fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f20740c3e40 0x7f20740678f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:34012/0 (socket says 172.21.15.118:34012) 2024-08-18T13:40:00.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20737fe700 1 -- 172.21.15.118:0/2410046988 learned_addr learned my addr 172.21.15.118:0/2410046988 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:00.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20727fc700 1 --2- 172.21.15.118:0/2410046988 >> v2:172.21.15.118:3300/0 conn(0x7f20740c4fc0 0x7f207414d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f2072ffd700 1 --2- 172.21.15.118:0/2410046988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c3a70 0x7f207414cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 --2- 172.21.15.118:0/1508380555 >> v2:172.21.15.178:3300/0 conn(0x7f594c153830 0x7f594c06d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 -- 172.21.15.118:0/1508380555 >> 172.21.15.118:0/1508380555 conn(0x7f594c0c44b0 msgr2=0x7f594c0c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 -- 172.21.15.118:0/1508380555 shutdown_connections 2024-08-18T13:40:00.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f5951968700 1 -- 172.21.15.118:0/1508380555 wait complete. 2024-08-18T13:40:00.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fa612330700 1 -- 172.21.15.118:0/1350837693 wait complete. 2024-08-18T13:40:00.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20727fc700 1 -- 172.21.15.118:0/2410046988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c3a70 msgr2=0x7f207414cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20727fc700 1 --2- 172.21.15.118:0/2410046988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c3a70 0x7f207414cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20727fc700 1 -- 172.21.15.118:0/2410046988 >> v1:172.21.15.178:6789/0 conn(0x7f20740c3e40 legacy=0x7f20740678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20727fc700 1 -- 172.21.15.118:0/2410046988 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074153490 con 0x7f20740c4fc0 2024-08-18T13:40:00.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe914d27700 1 -- 172.21.15.118:0/3464231294 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c1b80 msgr2=0x7fe91006e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fe914d27700 1 --2- 172.21.15.118:0/3464231294 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c1b80 0x7fe91006e030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe908009e60 tx=0x7fe908009780 comp rx=0 tx=0).stop 2024-08-18T13:40:00.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce12fd4700 1 -- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c155510 msgr2=0x7fce0c06ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce12fd4700 1 --2- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c155510 0x7fce0c06ec60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce04009f20 tx=0x7fce04009840 comp rx=0 tx=0).stop 2024-08-18T13:40:00.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce12fd4700 1 -- 172.21.15.118:0/3959293096 shutdown_connections 2024-08-18T13:40:00.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce12fd4700 1 --2- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c155510 0x7fce0c06ec60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f20727fc700 1 --2- 172.21.15.118:0/2410046988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c4fc0 0x7f207414d520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f20680098e0 tx=0x7f20680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b464efcbcc5444 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce12fd4700 1 --2- 172.21.15.118:0/3959293096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c14b5d0 0x7fce0c14b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.263 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7fce12fd4700 1 -- 172.21.15.118:0/3959293096 >> 172.21.15.118:0/3959293096 conn(0x7fce0c0c8660 msgr2=0x7fce0c0c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.263 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f85acda5700 1 Processor -- start 2024-08-18T13:40:00.263 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f2071ffb700 1 -- 172.21.15.118:0/2410046988 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068009e80 con 0x7f20740c4fc0 2024-08-18T13:40:00.263 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f2071ffb700 1 -- 172.21.15.118:0/2410046988 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206800b860 con 0x7f20740c4fc0 2024-08-18T13:40:00.263 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f2071ffb700 1 -- 172.21.15.118:0/2410046988 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206800bbe0 con 0x7f20740c4fc0 2024-08-18T13:40:00.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.242+0000 7f85acda5700 1 -- start start 2024-08-18T13:40:00.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 Processor -- start 2024-08-18T13:40:00.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 -- start start 2024-08-18T13:40:00.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 -- 172.21.15.118:0/2410046988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c4fc0 msgr2=0x7f207414d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 --2- 172.21.15.118:0/2410046988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c4fc0 0x7f207414d520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f20680098e0 tx=0x7f20680091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 Processor -- start 2024-08-18T13:40:00.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 -- 172.21.15.118:0/3959293096 shutdown_connections 2024-08-18T13:40:00.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 -- 172.21.15.118:0/3959293096 wait complete. 2024-08-18T13:40:00.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fe914d27700 1 -- 172.21.15.118:0/3464231294 shutdown_connections 2024-08-18T13:40:00.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fe914d27700 1 --2- 172.21.15.118:0/3464231294 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c1b80 0x7fe91006e030 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fe914d27700 1 -- 172.21.15.118:0/3464231294 >> 172.21.15.118:0/3464231294 conn(0x7fe9100c44b0 msgr2=0x7fe9100c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85acda5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85a80c8200 0x7f85a806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fe914d27700 1 -- 172.21.15.118:0/3464231294 shutdown_connections 2024-08-18T13:40:00.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85acda5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a806edf0 con 0x7f85a80673d0 2024-08-18T13:40:00.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fe914d27700 1 -- 172.21.15.118:0/3464231294 wait complete. 2024-08-18T13:40:00.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 -- start start 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594c06f3b0 0x7f594c0c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c153830 0x7f594c06b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594c153c00 0x7f594c16d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c153340 con 0x7f594c153830 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c152170 con 0x7f594c06f3b0 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f5951968700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c152e50 con 0x7f594c153c00 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594c06f3b0 0x7f594c0c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594b7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594c153c00 0x7f594c16d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594b7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594c153c00 0x7f594c16d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54374/0 (socket says 172.21.15.118:54374) 2024-08-18T13:40:00.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594b7fe700 1 -- 172.21.15.118:0/1021067127 learned_addr learned my addr 172.21.15.118:0/1021067127 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa60404dce0 0x7fa604049370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa60404fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85acda5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a815a0b0 con 0x7f85a8153690 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85acda5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a815a5f0 con 0x7f85a80c8200 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a77fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f85a8153690 0x7f85a80c22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:34026/0 (socket says 172.21.15.118:34026) 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a77fe700 1 -- 172.21.15.118:0/1150657222 learned_addr learned my addr 172.21.15.118:0/1150657222 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a7fff700 1 --2- 172.21.15.118:0/1150657222 >> v2:172.21.15.178:3300/0 conn(0x7f85a80c8200 0x7f85a806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a7fff700 1 -- 172.21.15.118:0/1150657222 >> v1:172.21.15.178:6789/0 conn(0x7f85a8153690 legacy=0x7f85a80c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a7fff700 1 -- 172.21.15.118:0/1150657222 >> v1:172.21.15.118:6789/0 conn(0x7f85a80673d0 legacy=0x7f85a80c29b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T13:40:00.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a7fff700 1 -- 172.21.15.118:0/1150657222 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8153d10 con 0x7f85a80c8200 2024-08-18T13:40:00.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a7fff700 1 --2- 172.21.15.118:0/1150657222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a80c8200 0x7f85a806f280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f859c009860 tx=0x7f859c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc7b2c3bc340effe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6040d8160 0x7fa6040d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa604047ec0 con 0x7fa6040d7d90 2024-08-18T13:40:00.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040353a0 con 0x7fa6040d8160 2024-08-18T13:40:00.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa612330700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa604035640 con 0x7fa60404dce0 2024-08-18T13:40:00.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 -- 172.21.15.118:0/2410046988 shutdown_connections 2024-08-18T13:40:00.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 --2- 172.21.15.118:0/2410046988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c4fc0 0x7f207414d520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 --2- 172.21.15.118:0/2410046988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c3a70 0x7f207414cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa60404fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa60404fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45228/0 (socket says 172.21.15.118:45228) 2024-08-18T13:40:00.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 -- 172.21.15.118:0/1495522988 learned_addr learned my addr 172.21.15.118:0/1495522988 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 -- 172.21.15.118:0/2410046988 >> 172.21.15.118:0/2410046988 conn(0x7f20740c44b0 msgr2=0x7f20740c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 -- 172.21.15.118:0/2410046988 shutdown_connections 2024-08-18T13:40:00.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f20793f8700 1 -- 172.21.15.118:0/2410046988 wait complete. 2024-08-18T13:40:00.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 Processor -- start 2024-08-18T13:40:00.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 -- start start 2024-08-18T13:40:00.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fe914d27700 1 Processor -- start 2024-08-18T13:40:00.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594affd700 1 -- 172.21.15.118:0/1021067127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594c153c00 msgr2=0x7f594c16d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594affd700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594c153c00 0x7f594c16d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594affd700 1 -- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c153830 msgr2=0x7f594c06b5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T13:40:00.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594affd700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c153830 0x7f594c06b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f594affd700 1 -- 172.21.15.118:0/1021067127 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009420 con 0x7f594c06f3b0 2024-08-18T13:40:00.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa61132e700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa60404dce0 0x7fa604049370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa611b2f700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6040d8160 0x7fa6040d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 -- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa60404dce0 msgr2=0x7fa604049370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa60404dce0 0x7fa604049370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 -- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6040d8160 msgr2=0x7fa6040d65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6040d8160 0x7fa6040d65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 -- 172.21.15.118:0/1495522988 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c04c770 con 0x7fa6040d7d90 2024-08-18T13:40:00.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa611b2f700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6040d8160 0x7fa6040d65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:00.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fa610b2d700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa60404fd80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00d890 tx=0x7fa5fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0c06db90 0x7fce0c0bed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c14b5d0 0x7fce0c142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c155510 0x7fce0c143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c14def0 con 0x7fce0c06db90 2024-08-18T13:40:00.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c14e190 con 0x7fce0c14b5d0 2024-08-18T13:40:00.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce12fd4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c142ae0 con 0x7fce0c155510 2024-08-18T13:40:00.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce10d70700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0c06db90 0x7fce0c0bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce10d70700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0c06db90 0x7fce0c0bed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45234/0 (socket says 172.21.15.118:45234) 2024-08-18T13:40:00.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7fce10d70700 1 -- 172.21.15.118:0/2434324738 learned_addr learned my addr 172.21.15.118:0/2434324738 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce0bfff700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c14b5d0 0x7fce0c142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f594affd700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594c06f3b0 0x7f594c0c60b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f594400ed60 tx=0x7f594400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce10d70700 1 -- 172.21.15.118:0/2434324738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c155510 msgr2=0x7fce0c143530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T13:40:00.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce10d70700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c155510 0x7fce0c143530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce10d70700 1 -- 172.21.15.118:0/2434324738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c14b5d0 msgr2=0x7fce0c142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce10d70700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c14b5d0 0x7fce0c142080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce10d70700 1 -- 172.21.15.118:0/2434324738 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04009ae0 con 0x7fce0c06db90 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 Processor -- start 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a67fc700 1 -- 172.21.15.118:0/1150657222 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c011630 con 0x7f85a80c8200 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.246+0000 7f85a67fc700 1 -- 172.21.15.118:0/1150657222 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c011790 con 0x7f85a80c8200 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f85acda5700 1 -- 172.21.15.118:0/1150657222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a80c8200 msgr2=0x7f85a806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f85acda5700 1 --2- 172.21.15.118:0/1150657222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a80c8200 0x7f85a806f280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f859c009860 tx=0x7f859c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 -- start start 2024-08-18T13:40:00.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce10d70700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0c06db90 0x7fce0c0bed30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c7d0 tx=0x7fcdfc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce08ff9700 1 -- 172.21.15.118:0/2434324738 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc011600 con 0x7fce0c06db90 2024-08-18T13:40:00.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce08ff9700 1 -- 172.21.15.118:0/2434324738 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc011760 con 0x7fce0c06db90 2024-08-18T13:40:00.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce12fd4700 1 -- 172.21.15.118:0/2434324738 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c141320 con 0x7fce0c06db90 2024-08-18T13:40:00.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c3a70 0x7f20741512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20740c3e40 0x7f20740c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c4fc0 0x7f20740c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740c6150 con 0x7f20740c3a70 2024-08-18T13:40:00.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207406e0e0 con 0x7f20740c3e40 2024-08-18T13:40:00.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740c5860 con 0x7f20740c4fc0 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe914d27700 1 -- start start 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe914d27700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe910068b40 0x7fe910148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe914d27700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c11a0 0x7fe910169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe914d27700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9100c1b80 0x7fe91016a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe914d27700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91016dc90 con 0x7fe9100c11a0 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe914d27700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91016ce30 con 0x7fe9100c1b80 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe914d27700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91016d080 con 0x7fe910068b40 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c11a0 0x7fe910169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90e59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe910068b40 0x7fe910148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c11a0 0x7fe910169b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45244/0 (socket says 172.21.15.118:45244) 2024-08-18T13:40:00.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90dd9b700 1 -- 172.21.15.118:0/3008033134 learned_addr learned my addr 172.21.15.118:0/3008033134 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20727fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20740c3e40 0x7f20740c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20737fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c4fc0 0x7f20740c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce08ff9700 1 -- 172.21.15.118:0/2434324738 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc011b20 con 0x7fce0c06db90 2024-08-18T13:40:00.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce12fd4700 1 -- 172.21.15.118:0/2434324738 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c0bf270 con 0x7fce0c06db90 2024-08-18T13:40:00.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90ed9d700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9100c1b80 0x7fe91016a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.282 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20737fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c4fc0 0x7f20740c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54396/0 (socket says 172.21.15.118:54396) 2024-08-18T13:40:00.282 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20737fe700 1 -- 172.21.15.118:0/1205276993 learned_addr learned my addr 172.21.15.118:0/1205276993 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.282 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f2072ffd700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c3a70 0x7f20741512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.282 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f5932ffd700 1 -- 172.21.15.118:0/1021067127 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440092e0 con 0x7f594c06f3b0 2024-08-18T13:40:00.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f2072ffd700 1 -- 172.21.15.118:0/1205276993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c4fc0 msgr2=0x7f20740c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f2072ffd700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c4fc0 0x7f20740c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f2072ffd700 1 -- 172.21.15.118:0/1205276993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20740c3e40 msgr2=0x7f20740c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f2072ffd700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20740c3e40 0x7f20740c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f2072ffd700 1 -- 172.21.15.118:0/1205276993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2068009420 con 0x7f20740c3a70 2024-08-18T13:40:00.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f5932ffd700 1 -- 172.21.15.118:0/1021067127 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594400f040 con 0x7f594c06f3b0 2024-08-18T13:40:00.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20737fe700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c4fc0 0x7f20740c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:00.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90ed9d700 1 -- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe910068b40 msgr2=0x7fe910148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90ed9d700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe910068b40 0x7fe910148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90ed9d700 1 -- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c11a0 msgr2=0x7fe910169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90ed9d700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c11a0 0x7fe910169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90ed9d700 1 -- 172.21.15.118:0/3008033134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908009a20 con 0x7fe9100c1b80 2024-08-18T13:40:00.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90dd9b700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c11a0 0x7fe910169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:00.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f2072ffd700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c3a70 0x7f20741512b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f206400d890 tx=0x7f2064008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce12fd4700 1 -- 172.21.15.118:0/2434324738 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c14bcc0 con 0x7fce0c06db90 2024-08-18T13:40:00.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fa6017fa700 1 -- 172.21.15.118:0/1495522988 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc009610 con 0x7fa6040d7d90 2024-08-18T13:40:00.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe90ed9d700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9100c1b80 0x7fe91016a0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe904001eb0 tx=0x7fe904007e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fa6017fa700 1 -- 172.21.15.118:0/1495522988 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc009770 con 0x7fa6040d7d90 2024-08-18T13:40:00.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fa6017fa700 1 -- 172.21.15.118:0/1495522988 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc009b30 con 0x7fa6040d7d90 2024-08-18T13:40:00.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fa612330700 1 -- 172.21.15.118:0/1495522988 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040476b0 con 0x7fa6040d7d90 2024-08-18T13:40:00.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fa612330700 1 -- 172.21.15.118:0/1495522988 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60404a360 con 0x7fa6040d7d90 2024-08-18T13:40:00.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f85acda5700 1 -- 172.21.15.118:0/1150657222 shutdown_connections 2024-08-18T13:40:00.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f85acda5700 1 --2- 172.21.15.118:0/1150657222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a80c8200 0x7f85a806f280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f85acda5700 1 -- 172.21.15.118:0/1150657222 >> 172.21.15.118:0/1150657222 conn(0x7f85a806c020 msgr2=0x7f85a80c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f205affd700 1 -- 172.21.15.118:0/1205276993 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064007260 con 0x7f20740c3a70 2024-08-18T13:40:00.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f205affd700 1 -- 172.21.15.118:0/1205276993 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20640125f0 con 0x7f20740c3a70 2024-08-18T13:40:00.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f205affd700 1 -- 172.21.15.118:0/1205276993 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640129b0 con 0x7f20740c3a70 2024-08-18T13:40:00.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2074163ad0 con 0x7f20740c3a70 2024-08-18T13:40:00.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f5932ffd700 1 -- 172.21.15.118:0/1021067127 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594400cb50 con 0x7f594c06f3b0 2024-08-18T13:40:00.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c16aa80 con 0x7f594c06f3b0 2024-08-18T13:40:00.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c16cd20 con 0x7f594c06f3b0 2024-08-18T13:40:00.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20740c5de0 con 0x7f20740c3a70 2024-08-18T13:40:00.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fce08ff9700 1 -- 172.21.15.118:0/2434324738 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcdfc018350 con 0x7fce0c06db90 2024-08-18T13:40:00.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fce08ff9700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdf8060860 0x7fcdf8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fce08ff9700 1 -- 172.21.15.118:0/2434324738 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fcdfc07ce30 con 0x7fce0c06db90 2024-08-18T13:40:00.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fce0bfff700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdf8060860 0x7fcdf8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.250+0000 7fe8fe7fc700 1 -- 172.21.15.118:0/3008033134 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90400ce40 con 0x7fe9100c1b80 2024-08-18T13:40:00.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe8fe7fc700 1 -- 172.21.15.118:0/3008033134 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe904005b30 con 0x7fe9100c1b80 2024-08-18T13:40:00.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe8fe7fc700 1 -- 172.21.15.118:0/3008033134 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040070d0 con 0x7fe9100c1b80 2024-08-18T13:40:00.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fce0bfff700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdf8060860 0x7fcdf8062d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce0c147630 tx=0x7fce00009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fa6017fa700 1 -- 172.21.15.118:0/1495522988 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa5fc01e460 con 0x7fa6040d7d90 2024-08-18T13:40:00.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fa6017fa700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa5f00609e0 0x7fa5f0062e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f85acda5700 1 -- 172.21.15.118:0/1150657222 shutdown_connections 2024-08-18T13:40:00.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f85acda5700 1 -- 172.21.15.118:0/1150657222 wait complete. 2024-08-18T13:40:00.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f5932ffd700 1 -- 172.21.15.118:0/1021067127 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5944014480 con 0x7f594c06f3b0 2024-08-18T13:40:00.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91016d360 con 0x7fe9100c1b80 2024-08-18T13:40:00.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91016d920 con 0x7fe9100c1b80 2024-08-18T13:40:00.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe910071480 con 0x7fe9100c1b80 2024-08-18T13:40:00.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe8fe7fc700 1 -- 172.21.15.118:0/3008033134 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe904007480 con 0x7fe9100c1b80 2024-08-18T13:40:00.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2074071480 con 0x7f20740c3a70 2024-08-18T13:40:00.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f205affd700 1 -- 172.21.15.118:0/1205276993 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2064012d20 con 0x7f20740c3a70 2024-08-18T13:40:00.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f34ba7e0700 1 Processor -- start 2024-08-18T13:40:00.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b926d6700 1 Processor -- start 2024-08-18T13:40:00.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b926d6700 1 -- start start 2024-08-18T13:40:00.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b926d6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8b8c0673d0 0x7f8b8c0c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b926d6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b8c0c8200 0x7f8b8c0c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b926d6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c06edf0 con 0x7f8b8c15bd80 2024-08-18T13:40:00.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b926d6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c162de0 con 0x7f8b8c0673d0 2024-08-18T13:40:00.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b926d6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c162890 con 0x7f8b8c0c8200 2024-08-18T13:40:00.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b916d4700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f8b8c15bd80 0x7f8b8c06f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45954/0 (socket says 172.21.15.118:45954) 2024-08-18T13:40:00.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b916d4700 1 -- 172.21.15.118:0/1677541004 learned_addr learned my addr 172.21.15.118:0/1677541004 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:00.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b91ed5700 1 --2- 172.21.15.118:0/1677541004 >> v2:172.21.15.178:3300/0 conn(0x7f8b8c0c8200 0x7f8b8c0c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f205affd700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2060060960 0x7f2060062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f34ba7e0700 1 -- start start 2024-08-18T13:40:00.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f85acda5700 1 Processor -- start 2024-08-18T13:40:00.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f85acda5700 1 -- start start 2024-08-18T13:40:00.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f5932ffd700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59380608b0 0x7f5938062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c16cf40 con 0x7f594c06f3b0 2024-08-18T13:40:00.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f5932ffd700 1 -- 172.21.15.118:0/1021067127 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f5944050f70 con 0x7f594c06f3b0 2024-08-18T13:40:00.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f594a7fc700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59380608b0 0x7f5938062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fa6017fa700 1 -- 172.21.15.118:0/1495522988 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fa5fc07d510 con 0x7fa6040d7d90 2024-08-18T13:40:00.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fa612330700 1 -- 172.21.15.118:0/1495522988 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6040357f0 con 0x7fa6040d7d90 2024-08-18T13:40:00.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe8fe7fc700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe8f00609b0 0x7fe8f0062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe8fe7fc700 1 -- 172.21.15.118:0/3008033134 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fe90407c110 con 0x7fe9100c1b80 2024-08-18T13:40:00.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fe90e59c700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe8f00609b0 0x7fe8f0062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f20727fc700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2060060960 0x7f2060062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7f8b90ed3700 1 --2- 172.21.15.118:0/1677541004 >> v2:172.21.15.118:3300/0 conn(0x7f8b8c0673d0 0x7f8b8c0c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f8b91ed5700 1 -- 172.21.15.118:0/1677541004 >> v2:172.21.15.118:3300/0 conn(0x7f8b8c0673d0 msgr2=0x7f8b8c0c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.254+0000 7fa61132e700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa5f00609e0 0x7fa5f0062e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f8b91ed5700 1 --2- 172.21.15.118:0/1677541004 >> v2:172.21.15.118:3300/0 conn(0x7f8b8c0673d0 0x7f8b8c0c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f8b91ed5700 1 -- 172.21.15.118:0/1677541004 >> v1:172.21.15.118:6789/0 conn(0x7f8b8c15bd80 legacy=0x7f8b8c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f8b91ed5700 1 -- 172.21.15.118:0/1677541004 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c15c500 con 0x7f8b8c0c8200 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f8b90ed3700 1 --2- 172.21.15.118:0/1677541004 >> v2:172.21.15.118:3300/0 conn(0x7f8b8c0673d0 0x7f8b8c0c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a80673d0 0x7f85a8159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85a80c8200 0x7f85a816c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8153690 0x7f85a8158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a80645d0 con 0x7f85a80c8200 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a8157610 con 0x7f85a8153690 2024-08-18T13:40:00.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a8157e90 con 0x7f85a80673d0 2024-08-18T13:40:00.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f594a7fc700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59380608b0 0x7f5938062d60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5940000f60 tx=0x7f5940007b40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7fa61132e700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa5f00609e0 0x7fa5f0062e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa604009180 tx=0x7fa60c06c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f8b91ed5700 1 --2- 172.21.15.118:0/1677541004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c27e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0c8980 tx=0x7f8b7c009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9208004b4e7919a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85a80c8200 0x7f85a816c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7fe90e59c700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe8f00609b0 0x7fe8f0062e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe9000037e0 tx=0x7fe900009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85a80c8200 0x7f85a816c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45274/0 (socket says 172.21.15.118:45274) 2024-08-18T13:40:00.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 -- 172.21.15.118:0/2453462897 learned_addr learned my addr 172.21.15.118:0/2453462897 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a77fe700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a80673d0 0x7f85a8159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f20727fc700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2060060960 0x7f2060062e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20740c5390 tx=0x7f2068017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a7fff700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8153690 0x7f85a8158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34ba7e0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b40be510 0x7f34b414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34ba7e0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f34b40c12e0 0x7f34b40c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.300 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34ba7e0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b40c0e50 con 0x7f34b414bcf0 2024-08-18T13:40:00.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34ba7e0700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b40bf820 con 0x7f34b40c12e0 2024-08-18T13:40:00.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34ba7e0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4067470 con 0x7f34b40be510 2024-08-18T13:40:00.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b97de700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f34b414bcf0 0x7f34b4155560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:34040/0 (socket says 172.21.15.118:34040) 2024-08-18T13:40:00.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b97de700 1 -- 172.21.15.118:0/122069795 learned_addr learned my addr 172.21.15.118:0/122069795 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:00.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b9fdf700 1 --2- 172.21.15.118:0/122069795 >> v2:172.21.15.178:3300/0 conn(0x7f34b40c12e0 0x7f34b40c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b8fdd700 1 --2- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b40be510 0x7f34b414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.301 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 -- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a80673d0 msgr2=0x7f85a8159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a80673d0 0x7f85a8159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 -- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8153690 msgr2=0x7f85a8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8153690 0x7f85a8158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 -- 172.21.15.118:0/2453462897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c009420 con 0x7f85a80c8200 2024-08-18T13:40:00.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a7fff700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8153690 0x7f85a8158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:00.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85a6ffd700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85a80c8200 0x7f85a816c450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c950 tx=0x7f85a0002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b9fdf700 1 -- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b40be510 msgr2=0x7f34b414c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:00.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b9fdf700 1 --2- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b40be510 0x7f34b414c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b9fdf700 1 -- 172.21.15.118:0/122069795 >> v1:172.21.15.178:6789/0 conn(0x7f34b414bcf0 legacy=0x7f34b4155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b9fdf700 1 -- 172.21.15.118:0/122069795 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4156340 con 0x7f34b40c12e0 2024-08-18T13:40:00.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f34b9fdf700 1 --2- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 0x7f34b40c1760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009e60 tx=0x7f34ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96f0259684e4c5d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f205affd700 1 -- 172.21.15.118:0/1205276993 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f2064051820 con 0x7f20740c3a70 2024-08-18T13:40:00.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85977fe700 1 -- 172.21.15.118:0/2453462897 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0012630 con 0x7f85a80c8200 2024-08-18T13:40:00.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7fce08ff9700 1 -- 172.21.15.118:0/2434324738 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fce0c14bcc0 con 0x7fce0c06db90 2024-08-18T13:40:00.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a816ce00 con 0x7f85a80c8200 2024-08-18T13:40:00.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.258+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a8157840 con 0x7f85a80c8200 2024-08-18T13:40:00.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.266+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a8071410 con 0x7f85a80c8200 2024-08-18T13:40:00.305 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.266+0000 7fa6017fa700 1 -- 172.21.15.118:0/1495522988 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa6040357f0 con 0x7fa6040d7d90 2024-08-18T13:40:00.305 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.266+0000 7fe8fe7fc700 1 -- 172.21.15.118:0/3008033134 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe90404c250 con 0x7fe9100c1b80 2024-08-18T13:40:00.305 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f85977fe700 1 -- 172.21.15.118:0/2453462897 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0006ba0 con 0x7f85a80c8200 2024-08-18T13:40:00.305 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f85977fe700 1 -- 172.21.15.118:0/2453462897 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0006d10 con 0x7f85a80c8200 2024-08-18T13:40:00.305 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f205affd700 1 -- 172.21.15.118:0/1205276993 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f206404dbb0 con 0x7f20740c3a70 2024-08-18T13:40:00.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f85977fe700 1 -- 172.21.15.118:0/2453462897 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f85a00129a0 con 0x7f85a80c8200 2024-08-18T13:40:00.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f8b83fff700 1 -- 172.21.15.118:0/1677541004 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0092e0 con 0x7f8b8c0c8200 2024-08-18T13:40:00.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f85977fe700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f858c060860 0x7f858c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f85a77fe700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f858c060860 0x7f858c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f8b83fff700 1 -- 172.21.15.118:0/1677541004 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c00b5c0 con 0x7f8b8c0c8200 2024-08-18T13:40:00.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f8b83fff700 1 -- 172.21.15.118:0/1677541004 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00b980 con 0x7f8b8c0c8200 2024-08-18T13:40:00.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f5932ffd700 1 -- 172.21.15.118:0/1021067127 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f594404cb70 con 0x7f594c06f3b0 2024-08-18T13:40:00.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f34abfff700 1 -- 172.21.15.118:0/122069795 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00e070 con 0x7f34b40c12e0 2024-08-18T13:40:00.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f34abfff700 1 -- 172.21.15.118:0/122069795 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac00bb60 con 0x7f34b40c12e0 2024-08-18T13:40:00.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f34abfff700 1 -- 172.21.15.118:0/122069795 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac012430 con 0x7f34b40c12e0 2024-08-18T13:40:00.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f85977fe700 1 -- 172.21.15.118:0/2453462897 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f85a007d0d0 con 0x7f85a80c8200 2024-08-18T13:40:00.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.270+0000 7f85a77fe700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f858c060860 0x7f858c062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8598007180 tx=0x7f8598007eb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 -- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 msgr2=0x7f34b40c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 --2- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 0x7f34b40c1760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009e60 tx=0x7f34ac009780 comp rx=0 tx=0).stop 2024-08-18T13:40:00.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 -- 172.21.15.118:0/122069795 shutdown_connections 2024-08-18T13:40:00.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 --2- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 0x7f34b40c1760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 --2- 172.21.15.118:0/122069795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b40be510 0x7f34b414c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 -- 172.21.15.118:0/122069795 >> 172.21.15.118:0/122069795 conn(0x7f34b406c020 msgr2=0x7f34b40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 -- 172.21.15.118:0/122069795 shutdown_connections 2024-08-18T13:40:00.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 -- 172.21.15.118:0/122069795 wait complete. 2024-08-18T13:40:00.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 Processor -- start 2024-08-18T13:40:00.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 -- start start 2024-08-18T13:40:00.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.274+0000 7f34ba7e0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34b40be510 0x7f34b41652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f85977fe700 1 -- 172.21.15.118:0/2453462897 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f85a004d230 con 0x7f85a80c8200 2024-08-18T13:40:00.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 -- 172.21.15.118:0/1677541004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0c8200 msgr2=0x7f8b8c0c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 --2- 172.21.15.118:0/1677541004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c27e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0c8980 tx=0x7f8b7c009670 comp rx=0 tx=0).stop 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34ba7e0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 0x7f34b4168710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34ba7e0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b414bcf0 0x7f34b40bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34ba7e0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40c76a0 con 0x7f34b40be510 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34ba7e0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4168280 con 0x7f34b40c12e0 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34ba7e0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4148320 con 0x7f34b414bcf0 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34b40be510 0x7f34b41652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34b40be510 0x7f34b41652a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45288/0 (socket says 172.21.15.118:45288) 2024-08-18T13:40:00.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 -- 172.21.15.118:0/435471351 learned_addr learned my addr 172.21.15.118:0/435471351 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b9fdf700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b414bcf0 0x7f34b40bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b8fdd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 0x7f34b4168710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 -- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b414bcf0 msgr2=0x7f34b40bffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b414bcf0 0x7f34b40bffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 -- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 msgr2=0x7f34b4168710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 0x7f34b4168710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 -- 172.21.15.118:0/435471351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac009a20 con 0x7f34b40be510 2024-08-18T13:40:00.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34b97de700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34b40be510 0x7f34b41652a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34b0001da0 tx=0x7f34b0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 -- 172.21.15.118:0/1677541004 shutdown_connections 2024-08-18T13:40:00.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 --2- 172.21.15.118:0/1677541004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c27e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 --2- 172.21.15.118:0/1677541004 >> v2:172.21.15.118:3300/0 conn(0x7f8b8c0673d0 0x7f8b8c0c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 -- 172.21.15.118:0/1677541004 >> 172.21.15.118:0/1677541004 conn(0x7f8b8c06c020 msgr2=0x7f8b8c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34a97fa700 1 -- 172.21.15.118:0/435471351 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0005b30 con 0x7f34b40be510 2024-08-18T13:40:00.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 -- 172.21.15.118:0/1677541004 shutdown_connections 2024-08-18T13:40:00.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34ba7e0700 1 -- 172.21.15.118:0/435471351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b4143130 con 0x7f34b40be510 2024-08-18T13:40:00.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34ba7e0700 1 -- 172.21.15.118:0/435471351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b40c70e0 con 0x7f34b40be510 2024-08-18T13:40:00.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34a97fa700 1 -- 172.21.15.118:0/435471351 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b0005c90 con 0x7f34b40be510 2024-08-18T13:40:00.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f34a97fa700 1 -- 172.21.15.118:0/435471351 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0008aa0 con 0x7f34b40be510 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 -- 172.21.15.118:0/1677541004 wait complete. 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 Processor -- start 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.278+0000 7f8b926d6700 1 -- start start 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b926d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0673d0 0x7f8b8c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b926d6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b926d6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b8c15bd80 0x7f8b8c164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b926d6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0bf050 con 0x7f8b8c0c8200 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b926d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c1625b0 con 0x7f8b8c0673d0 2024-08-18T13:40:00.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b926d6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c162120 con 0x7f8b8c15bd80 2024-08-18T13:40:00.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f34a97fa700 1 -- 172.21.15.118:0/435471351 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f34b0012080 con 0x7f34b40be510 2024-08-18T13:40:00.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b916d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0673d0 0x7f8b8c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45304/0 (socket says 172.21.15.118:45304) 2024-08-18T13:40:00.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 -- 172.21.15.118:0/3473224206 learned_addr learned my addr 172.21.15.118:0/3473224206 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:00.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b91ed5700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b8c15bd80 0x7f8b8c164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 -- 172.21.15.118:0/3473224206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b8c15bd80 msgr2=0x7f8b8c164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b8c15bd80 0x7f8b8c164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 -- 172.21.15.118:0/3473224206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0673d0 msgr2=0x7f8b8c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0673d0 0x7f8b8c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 -- 172.21.15.118:0/3473224206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c009910 con 0x7f8b8c0c8200 2024-08-18T13:40:00.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f34a97fa700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f34980609b0 0x7f3498062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f8b90ed3700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c01d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400dbb0 tx=0x7f8b84009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f34b8fdd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f34980609b0 0x7f3498062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f34a97fa700 1 -- 172.21.15.118:0/435471351 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f34b008d4b0 con 0x7f34b40be510 2024-08-18T13:40:00.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.282+0000 7f34b8fdd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f34980609b0 0x7f3498062e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34b406dda0 tx=0x7f34a40091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.290+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34940052b0 con 0x7f34b40be510 2024-08-18T13:40:00.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.290+0000 7f8b817fa700 1 -- 172.21.15.118:0/3473224206 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400b8a0 con 0x7f8b8c0c8200 2024-08-18T13:40:00.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.290+0000 7f8b817fa700 1 -- 172.21.15.118:0/3473224206 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8400ba00 con 0x7f8b8c0c8200 2024-08-18T13:40:00.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.290+0000 7f8b817fa700 1 -- 172.21.15.118:0/3473224206 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400bdc0 con 0x7f8b8c0c8200 2024-08-18T13:40:00.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.290+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c161b70 con 0x7f8b8c0c8200 2024-08-18T13:40:00.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.290+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c197e40 con 0x7f8b8c0c8200 2024-08-18T13:40:00.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.290+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c1622d0 con 0x7f8b8c0c8200 2024-08-18T13:40:00.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.294+0000 7f34a97fa700 1 -- 172.21.15.118:0/435471351 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f34b005d5f0 con 0x7f34b40be510 2024-08-18T13:40:00.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.294+0000 7f8b817fa700 1 -- 172.21.15.118:0/3473224206 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8b84012600 con 0x7f8b8c0c8200 2024-08-18T13:40:00.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.298+0000 7f8b817fa700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8b700609e0 0x7f8b70062e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:00.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.298+0000 7f8b817fa700 1 -- 172.21.15.118:0/3473224206 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f8b8407d160 con 0x7f8b8c0c8200 2024-08-18T13:40:00.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.298+0000 7f8b916d4700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8b700609e0 0x7f8b70062e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:00.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.298+0000 7f8b916d4700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8b700609e0 0x7f8b70062e90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b88001950 tx=0x7f8b88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:00.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.306+0000 7f8b817fa700 1 -- 172.21.15.118:0/3473224206 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8b8404d320 con 0x7f8b8c0c8200 2024-08-18T13:40:00.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.562+0000 7fce12fd4700 1 -- 172.21.15.118:0/2434324738 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fce0c14bcc0 con 0x7fce0c06db90 2024-08-18T13:40:00.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.562+0000 7fce08ff9700 1 -- 172.21.15.118:0/2434324738 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcdfc01e020 con 0x7fce0c06db90 2024-08-18T13:40:00.568 INFO:teuthology.orchestra.run.smithi118.stdout:60129542153 2024-08-18T13:40:00.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 -- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdf8060860 msgr2=0x7fcdf8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdf8060860 0x7fcdf8062d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce0c147630 tx=0x7fce00009060 comp rx=0 tx=0).stop 2024-08-18T13:40:00.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 -- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0c06db90 msgr2=0x7fce0c0bed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0c06db90 0x7fce0c0bed30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c7d0 tx=0x7fcdfc008040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 -- 172.21.15.118:0/2434324738 shutdown_connections 2024-08-18T13:40:00.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdf8060860 0x7fcdf8062d10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce0c155510 0x7fce0c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce0c14b5d0 0x7fce0c142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 --2- 172.21.15.118:0/2434324738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fce0c06db90 0x7fce0c0bed30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 -- 172.21.15.118:0/2434324738 >> 172.21.15.118:0/2434324738 conn(0x7fce0c0c8660 msgr2=0x7fce0c15b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 -- 172.21.15.118:0/2434324738 shutdown_connections 2024-08-18T13:40:00.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.566+0000 7fcdf27fc700 1 -- 172.21.15.118:0/2434324738 wait complete. 2024-08-18T13:40:00.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.570+0000 7fa612330700 1 -- 172.21.15.118:0/1495522988 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa6040357f0 con 0x7fa6040d7d90 2024-08-18T13:40:00.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.570+0000 7fa6017fa700 1 -- 172.21.15.118:0/1495522988 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6040357f0 con 0x7fa6040d7d90 2024-08-18T13:40:00.577 INFO:teuthology.orchestra.run.smithi118.stdout:60129542153 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 -- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa5f00609e0 msgr2=0x7fa5f0062e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa5f00609e0 0x7fa5f0062e90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa604009180 tx=0x7fa60c06c000 comp rx=0 tx=0).stop 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 -- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 msgr2=0x7fa60404fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa60404fd80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00d890 tx=0x7fa5fc008040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 -- 172.21.15.118:0/1495522988 shutdown_connections 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa5f00609e0 0x7fa5f0062e90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6040d8160 0x7fa6040d65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6040d7d90 0x7fa60404fd80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 --2- 172.21.15.118:0/1495522988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa60404dce0 0x7fa604049370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 -- 172.21.15.118:0/1495522988 >> 172.21.15.118:0/1495522988 conn(0x7fa60400c6d0 msgr2=0x7fa604035ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 -- 172.21.15.118:0/1495522988 shutdown_connections 2024-08-18T13:40:00.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.574+0000 7fa5f6ffd700 1 -- 172.21.15.118:0/1495522988 wait complete. 2024-08-18T13:40:00.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.578+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f20740c1d70 con 0x7f20740c3a70 2024-08-18T13:40:00.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.578+0000 7f205affd700 1 -- 172.21.15.118:0/1205276993 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f206404dd50 con 0x7f20740c3a70 2024-08-18T13:40:00.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.0 2024-08-18T13:40:00.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.578+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe9101698b0 con 0x7fe9100c1b80 2024-08-18T13:40:00.585 INFO:teuthology.orchestra.run.smithi118.stdout:64424509451 2024-08-18T13:40:00.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe8fe7fc700 1 -- 172.21.15.118:0/3008033134 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe9101698b0 con 0x7fe9100c1b80 2024-08-18T13:40:00.586 INFO:teuthology.orchestra.run.smithi118.stdout:64424509450 2024-08-18T13:40:00.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2060060960 msgr2=0x7f2060062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2060060960 0x7f2060062e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20740c5390 tx=0x7f2068017000 comp rx=0 tx=0).stop 2024-08-18T13:40:00.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c3a70 msgr2=0x7f20741512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c3a70 0x7f20741512b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f206400d890 tx=0x7f2064008040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe8f00609b0 msgr2=0x7fe8f0062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 shutdown_connections 2024-08-18T13:40:00.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2060060960 0x7f2060062e10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20740c4fc0 0x7f20740c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20740c3e40 0x7f20740c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 --2- 172.21.15.118:0/1205276993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f20740c3a70 0x7f20741512b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 >> 172.21.15.118:0/1205276993 conn(0x7f20740c44b0 msgr2=0x7f207406e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe8f00609b0 0x7fe8f0062e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe9000037e0 tx=0x7fe900009040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 shutdown_connections 2024-08-18T13:40:00.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9100c1b80 msgr2=0x7fe91016a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9100c1b80 0x7fe91016a0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe904001eb0 tx=0x7fe904007e60 comp rx=0 tx=0).stop 2024-08-18T13:40:00.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7f20793f8700 1 -- 172.21.15.118:0/1205276993 wait complete. 2024-08-18T13:40:00.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 shutdown_connections 2024-08-18T13:40:00.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe8f00609b0 0x7fe8f0062e60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9100c1b80 0x7fe91016a0d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe9100c11a0 0x7fe910169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 --2- 172.21.15.118:0/3008033134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe910068b40 0x7fe910148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 >> 172.21.15.118:0/3008033134 conn(0x7fe9100c44b0 msgr2=0x7fe9100c2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 shutdown_connections 2024-08-18T13:40:00.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.582+0000 7fe914d27700 1 -- 172.21.15.118:0/3008033134 wait complete. 2024-08-18T13:40:00.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.586+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f594c16cb80 con 0x7f594c06f3b0 2024-08-18T13:40:00.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.586+0000 7f5932ffd700 1 -- 172.21.15.118:0/1021067127 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f594c16cb80 con 0x7f594c06f3b0 2024-08-18T13:40:00.592 INFO:teuthology.orchestra.run.smithi118.stdout:55834574856 2024-08-18T13:40:00.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.1 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59380608b0 msgr2=0x7f5938062d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59380608b0 0x7f5938062d60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5940000f60 tx=0x7f5940007b40 comp rx=0 tx=0).stop 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594c06f3b0 msgr2=0x7f594c0c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594c06f3b0 0x7f594c0c60b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f594400ed60 tx=0x7f594400b040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 shutdown_connections 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59380608b0 0x7f5938062d60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f594c153c00 0x7f594c16d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f594c153830 0x7f594c06b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 --2- 172.21.15.118:0/1021067127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f594c06f3b0 0x7f594c0c60b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 >> 172.21.15.118:0/1021067127 conn(0x7f594c0c44b0 msgr2=0x7f594c158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 shutdown_connections 2024-08-18T13:40:00.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.590+0000 7f5951968700 1 -- 172.21.15.118:0/1021067127 wait complete. 2024-08-18T13:40:00.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.7 2024-08-18T13:40:00.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.6 2024-08-18T13:40:00.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.598+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f85a80c7fc0 con 0x7f85a80c8200 2024-08-18T13:40:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85977fe700 1 -- 172.21.15.118:0/2453462897 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f85a0050850 con 0x7f85a80c8200 2024-08-18T13:40:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3494005560 con 0x7f34b40be510 2024-08-18T13:40:00.605 INFO:teuthology.orchestra.run.smithi118.stdout:64424509449 2024-08-18T13:40:00.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f34a97fa700 1 -- 172.21.15.118:0/435471351 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34b0060c10 con 0x7f34b40be510 2024-08-18T13:40:00.606 INFO:teuthology.orchestra.run.smithi118.stdout:60129542153 2024-08-18T13:40:00.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-08-18T13:40:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f34980609b0 msgr2=0x7f3498062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f34980609b0 0x7f3498062e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34b406dda0 tx=0x7f34a40091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34b40be510 msgr2=0x7f34b41652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34b40be510 0x7f34b41652a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34b0001da0 tx=0x7f34b0009040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 shutdown_connections 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f34980609b0 0x7f3498062e60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34b414bcf0 0x7f34b40bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34b40c12e0 0x7f34b4168710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 --2- 172.21.15.118:0/435471351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34b40be510 0x7f34b41652a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 >> 172.21.15.118:0/435471351 conn(0x7f34b406c020 msgr2=0x7f34b414b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 shutdown_connections 2024-08-18T13:40:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f349effd700 1 -- 172.21.15.118:0/435471351 wait complete. 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f858c060860 msgr2=0x7f858c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f858c060860 0x7f858c062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8598007180 tx=0x7f8598007eb0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85a80c8200 msgr2=0x7f85a816c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85a80c8200 0x7f85a816c450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c950 tx=0x7f85a0002760 comp rx=0 tx=0).stop 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 shutdown_connections 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f858c060860 0x7f858c062d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8153690 0x7f85a8158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f85a80c8200 0x7f85a816c450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 --2- 172.21.15.118:0/2453462897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a80673d0 0x7f85a8159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 >> 172.21.15.118:0/2453462897 conn(0x7f85a806c020 msgr2=0x7f85a80c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 shutdown_connections 2024-08-18T13:40:00.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.602+0000 7f85acda5700 1 -- 172.21.15.118:0/2453462897 wait complete. 2024-08-18T13:40:00.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.610+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8b8c1622d0 con 0x7f8b8c0c8200 2024-08-18T13:40:00.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b817fa700 1 -- 172.21.15.118:0/3473224206 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b8401f020 con 0x7f8b8c0c8200 2024-08-18T13:40:00.617 INFO:teuthology.orchestra.run.smithi118.stdout:60129542154 2024-08-18T13:40:00.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.2 2024-08-18T13:40:00.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8b700609e0 msgr2=0x7f8b70062e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8b700609e0 0x7f8b70062e90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8b88001950 tx=0x7f8b88009040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b8c0c8200 msgr2=0x7f8b8c0c01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c01d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400dbb0 tx=0x7f8b84009040 comp rx=0 tx=0).stop 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 shutdown_connections 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8b700609e0 0x7f8b70062e90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b8c15bd80 0x7f8b8c164fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b8c0c8200 0x7f8b8c0c01d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 --2- 172.21.15.118:0/3473224206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b8c0673d0 0x7f8b8c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 >> 172.21.15.118:0/3473224206 conn(0x7f8b8c06c020 msgr2=0x7f8b8c168520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 shutdown_connections 2024-08-18T13:40:00.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:00.614+0000 7f8b926d6700 1 -- 172.21.15.118:0/3473224206 wait complete. 2024-08-18T13:40:00.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.4 2024-08-18T13:40:00.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.3 2024-08-18T13:40:01.585 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-18T13:40:01.595 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-18T13:40:01.600 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-18T13:40:01.603 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-18T13:40:01.608 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-18T13:40:01.619 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-18T13:40:01.623 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-18T13:40:01.630 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-18T13:40:01.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee3a36700 1 Processor -- start 2024-08-18T13:40:01.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee3a36700 1 -- start start 2024-08-18T13:40:01.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee3a36700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc14b520 0x7f9edc155400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee3a36700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9edc0c57c0 0x7f9edc0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee3a36700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9edc0c6600 con 0x7f9edc14b8f0 2024-08-18T13:40:01.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee3a36700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0c7460 con 0x7f9edc0c57c0 2024-08-18T13:40:01.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee3a36700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0c6850 con 0x7f9edc14b520 2024-08-18T13:40:01.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc14b520 0x7f9edc155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc14b520 0x7f9edc155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54446/0 (socket says 172.21.15.118:54446) 2024-08-18T13:40:01.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 -- 172.21.15.118:0/3090793640 learned_addr learned my addr 172.21.15.118:0/3090793640 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee0fd1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9edc0c57c0 0x7f9edc0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 -- 172.21.15.118:0/3090793640 >> v2:172.21.15.118:3300/0 conn(0x7f9edc0c57c0 msgr2=0x7f9edc0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:01.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 --2- 172.21.15.118:0/3090793640 >> v2:172.21.15.118:3300/0 conn(0x7f9edc0c57c0 0x7f9edc0673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 -- 172.21.15.118:0/3090793640 >> v1:172.21.15.178:6789/0 conn(0x7f9edc14b8f0 legacy=0x7f9edc0c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 -- 172.21.15.118:0/3090793640 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0c6b30 con 0x7f9edc14b520 2024-08-18T13:40:01.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee0fd1700 1 --2- 172.21.15.118:0/3090793640 >> v2:172.21.15.118:3300/0 conn(0x7f9edc0c57c0 0x7f9edc0673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:40:01.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.726+0000 7f9ee17d2700 1 --2- 172.21.15.118:0/3090793640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc14b520 0x7f9edc155400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009f20 tx=0x7f9ed4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96432f9bd0f2390 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ed3fff700 1 -- 172.21.15.118:0/3090793640 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40092e0 con 0x7f9edc14b520 2024-08-18T13:40:01.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ed3fff700 1 -- 172.21.15.118:0/3090793640 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4011470 con 0x7f9edc14b520 2024-08-18T13:40:01.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ed3fff700 1 -- 172.21.15.118:0/3090793640 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4011810 con 0x7f9edc14b520 2024-08-18T13:40:01.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- 172.21.15.118:0/3090793640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc14b520 msgr2=0x7f9edc155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 --2- 172.21.15.118:0/3090793640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc14b520 0x7f9edc155400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009f20 tx=0x7f9ed4009840 comp rx=0 tx=0).stop 2024-08-18T13:40:01.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- 172.21.15.118:0/3090793640 shutdown_connections 2024-08-18T13:40:01.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 --2- 172.21.15.118:0/3090793640 >> v2:172.21.15.118:3300/0 conn(0x7f9edc0c57c0 0x7f9edc0673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 --2- 172.21.15.118:0/3090793640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc14b520 0x7f9edc155400 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009f20 tx=0x7f9ed4009840 comp rx=0 tx=0).stop 2024-08-18T13:40:01.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- 172.21.15.118:0/3090793640 >> 172.21.15.118:0/3090793640 conn(0x7f9edc0c44b0 msgr2=0x7f9edc0c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- 172.21.15.118:0/3090793640 shutdown_connections 2024-08-18T13:40:01.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- 172.21.15.118:0/3090793640 wait complete. 2024-08-18T13:40:01.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 Processor -- start 2024-08-18T13:40:01.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- start start 2024-08-18T13:40:01.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9edc0c57c0 0x7f9edc165000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9edc14b8f0 0x7f9edc163770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc163cb0 0x7f9edc175910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc168b10 con 0x7f9edc0c57c0 2024-08-18T13:40:01.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc155d80 con 0x7f9edc14b8f0 2024-08-18T13:40:01.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee3a36700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc172530 con 0x7f9edc163cb0 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.730+0000 7f9ee17d2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9edc0c57c0 0x7f9edc165000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee1fd3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc163cb0 0x7f9edc175910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9edc0c57c0 0x7f9edc165000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45320/0 (socket says 172.21.15.118:45320) 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 -- 172.21.15.118:0/1008516067 learned_addr learned my addr 172.21.15.118:0/1008516067 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee0fd1700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9edc14b8f0 0x7f9edc163770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 -- 172.21.15.118:0/1008516067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc163cb0 msgr2=0x7f9edc175910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc163cb0 0x7f9edc175910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 -- 172.21.15.118:0/1008516067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9edc14b8f0 msgr2=0x7f9edc163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9edc14b8f0 0x7f9edc163770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 -- 172.21.15.118:0/1008516067 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4009ae0 con 0x7f9edc0c57c0 2024-08-18T13:40:01.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee17d2700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9edc0c57c0 0x7f9edc165000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0c6990 tx=0x7f9ed4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ed17fa700 1 -- 172.21.15.118:0/1008516067 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400b320 con 0x7f9edc0c57c0 2024-08-18T13:40:01.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ed17fa700 1 -- 172.21.15.118:0/1008516067 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed400b4a0 con 0x7f9edc0c57c0 2024-08-18T13:40:01.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ed17fa700 1 -- 172.21.15.118:0/1008516067 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed401d430 con 0x7f9edc0c57c0 2024-08-18T13:40:01.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee3a36700 1 -- 172.21.15.118:0/1008516067 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc172810 con 0x7f9edc0c57c0 2024-08-18T13:40:01.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.734+0000 7f9ee3a36700 1 -- 172.21.15.118:0/1008516067 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc172dd0 con 0x7f9edc0c57c0 2024-08-18T13:40:01.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.738+0000 7f9ed17fa700 1 -- 172.21.15.118:0/1008516067 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9ed401d820 con 0x7f9edc0c57c0 2024-08-18T13:40:01.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.738+0000 7f9ed17fa700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9ec0060930 0x7f9ec0062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.738+0000 7f9ed17fa700 1 -- 172.21.15.118:0/1008516067 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f9ed407f520 con 0x7f9edc0c57c0 2024-08-18T13:40:01.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.738+0000 7f9ee0fd1700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9ec0060930 0x7f9ec0062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.738+0000 7f9ee0fd1700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9ec0060930 0x7f9ec0062de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9edc164ca0 tx=0x7f9ecc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.738+0000 7f9ee3a36700 1 -- 172.21.15.118:0/1008516067 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc071480 con 0x7f9edc0c57c0 2024-08-18T13:40:01.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.750+0000 7f9ed17fa700 1 -- 172.21.15.118:0/1008516067 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9ed404f680 con 0x7f9edc0c57c0 2024-08-18T13:40:01.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.762+0000 7f79a3fff700 1 Processor -- start 2024-08-18T13:40:01.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.762+0000 7facd14e7700 1 Processor -- start 2024-08-18T13:40:01.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.762+0000 7facd14e7700 1 -- start start 2024-08-18T13:40:01.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.762+0000 7f79a3fff700 1 -- start start 2024-08-18T13:40:01.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729ae33700 1 Processor -- start 2024-08-18T13:40:01.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729ae33700 1 -- start start 2024-08-18T13:40:01.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79a40c8410 0x7f79a40c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a40c2ae0 con 0x7f79a40673d0 2024-08-18T13:40:01.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a406f720 con 0x7f79a414b630 2024-08-18T13:40:01.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a4067810 con 0x7f79a40c8410 2024-08-18T13:40:01.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7facd14e7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faccc152370 0x7faccc0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7facd14e7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0c14c0 con 0x7faccc0c57c0 2024-08-18T13:40:01.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7facd14e7700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc067570 con 0x7faccc0c4fc0 2024-08-18T13:40:01.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7facd14e7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc06d350 con 0x7faccc152370 2024-08-18T13:40:01.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7faccaffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faccc152370 0x7faccc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7faccb7fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7faccc0c57c0 0x7faccc14bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45960/0 (socket says 172.21.15.118:45960) 2024-08-18T13:40:01.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7faccb7fe700 1 -- 172.21.15.118:0/1695153401 learned_addr learned my addr 172.21.15.118:0/1695153401 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:01.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a27fc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f79a40673d0 0x7f79a4159c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45972/0 (socket says 172.21.15.118:45972) 2024-08-18T13:40:01.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a27fc700 1 -- 172.21.15.118:0/4117117481 learned_addr learned my addr 172.21.15.118:0/4117117481 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:01.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a37fe700 1 --2- 172.21.15.118:0/4117117481 >> v2:172.21.15.178:3300/0 conn(0x7f79a40c8410 0x7f79a40c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922857252 0 0) 0x7f79a40c2ae0 con 0x7f79a40673d0 2024-08-18T13:40:01.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f798c003570 con 0x7f79a40673d0 2024-08-18T13:40:01.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7faccaffd700 1 -- 172.21.15.118:0/1695153401 >> v1:172.21.15.178:6789/0 conn(0x7faccc0c4fc0 legacy=0x7faccc152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7faccaffd700 1 -- 172.21.15.118:0/1695153401 >> v1:172.21.15.118:6789/0 conn(0x7faccc0c57c0 legacy=0x7faccc14bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7faccaffd700 1 -- 172.21.15.118:0/1695153401 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc067850 con 0x7faccc152370 2024-08-18T13:40:01.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96f0cbe700 1 Processor -- start 2024-08-18T13:40:01.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96f0cbe700 1 -- start start 2024-08-18T13:40:01.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071528952 0 0) 0x7f79a406f720 con 0x7f79a414b630 2024-08-18T13:40:01.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a40c2ae0 con 0x7f79a414b630 2024-08-18T13:40:01.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a37fe700 1 -- 172.21.15.118:0/4117117481 >> v1:172.21.15.178:6789/0 conn(0x7f79a414b630 legacy=0x7f79a406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a37fe700 1 -- 172.21.15.118:0/4117117481 >> v1:172.21.15.118:6789/0 conn(0x7f79a40673d0 legacy=0x7f79a4159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 294875478 0 0) 0x7f798c003570 con 0x7f79a40673d0 2024-08-18T13:40:01.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a37fe700 1 -- 172.21.15.118:0/4117117481 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a4155430 con 0x7f79a40c8410 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729ae33700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f72940c7fc0 0x7f72940673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729ae33700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f729406f490 0x7f7294155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729ae33700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72940c2f20 con 0x7f72940c2440 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729ae33700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72941561f0 con 0x7f72940c7fc0 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729ae33700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729406fa30 con 0x7f729406f490 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f72940c7fc0 0x7f72940673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f72940c7fc0 0x7f72940673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45338/0 (socket says 172.21.15.118:45338) 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 -- 172.21.15.118:0/1336497333 learned_addr learned my addr 172.21.15.118:0/1336497333 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f729a632700 1 --2- 172.21.15.118:0/1336497333 >> v2:172.21.15.178:3300/0 conn(0x7f729406f490 0x7f7294155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a37fe700 1 --2- 172.21.15.118:0/4117117481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a40c8410 0x7f79a40c2f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f799400aa90 tx=0x7f799400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f162b5ab49828c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 -- 172.21.15.118:0/1336497333 >> v2:172.21.15.178:3300/0 conn(0x7f729406f490 msgr2=0x7f7294155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:01.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 --2- 172.21.15.118:0/1336497333 >> v2:172.21.15.178:3300/0 conn(0x7f729406f490 0x7f7294155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 -- 172.21.15.118:0/1336497333 >> v1:172.21.15.178:6789/0 conn(0x7f72940c2440 legacy=0x7f72940c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 -- 172.21.15.118:0/1336497333 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940c8490 con 0x7f72940c7fc0 2024-08-18T13:40:01.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998004cd0 con 0x7f79a40c8410 2024-08-18T13:40:01.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799400aed0 con 0x7f79a40c8410 2024-08-18T13:40:01.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a1ffb700 1 -- 172.21.15.118:0/4117117481 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799400c870 con 0x7f79a40c8410 2024-08-18T13:40:01.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7299630700 1 --2- 172.21.15.118:0/1336497333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f72940673d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f729406ca20 tx=0x7f7290009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537052ca18d6a893 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7298e2f700 1 -- 172.21.15.118:0/1336497333 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290009e70 con 0x7f72940c7fc0 2024-08-18T13:40:01.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96f0cbe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96ec14b5d0 0x7f96ec14b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96f0cbe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f96ec06ec60 0x7f96ec06f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96f0cbe700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec06a310 con 0x7f96ec155510 2024-08-18T13:40:01.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96f0cbe700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec06a800 con 0x7f96ec06ec60 2024-08-18T13:40:01.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96f0cbe700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec0c3120 con 0x7f96ec14b5d0 2024-08-18T13:40:01.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7298e2f700 1 -- 172.21.15.118:0/1336497333 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729000b860 con 0x7f72940c7fc0 2024-08-18T13:40:01.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f7298e2f700 1 -- 172.21.15.118:0/1336497333 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000bc00 con 0x7f72940c7fc0 2024-08-18T13:40:01.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7faccaffd700 1 --2- 172.21.15.118:0/1695153401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc152370 0x7faccc0c1950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7facc00098e0 tx=0x7facc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51eca1ef4545b85d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7facc9ffb700 1 -- 172.21.15.118:0/1695153401 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0009e80 con 0x7faccc152370 2024-08-18T13:40:01.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ea59c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96ec14b5d0 0x7f96ec14b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ead9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f96ec06ec60 0x7f96ec06f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ea59c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96ec14b5d0 0x7f96ec14b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:44100/0 (socket says 172.21.15.118:44100) 2024-08-18T13:40:01.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ea59c700 1 -- 172.21.15.118:0/140165587 learned_addr learned my addr 172.21.15.118:0/140165587 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7facc9ffb700 1 -- 172.21.15.118:0/1695153401 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc000b860 con 0x7faccc152370 2024-08-18T13:40:01.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7facc9ffb700 1 -- 172.21.15.118:0/1695153401 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000bbe0 con 0x7faccc152370 2024-08-18T13:40:01.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 -- 172.21.15.118:0/4117117481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a40c8410 msgr2=0x7f79a40c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 --2- 172.21.15.118:0/4117117481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a40c8410 0x7f79a40c2f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f799400aa90 tx=0x7f799400a330 comp rx=0 tx=0).stop 2024-08-18T13:40:01.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 -- 172.21.15.118:0/4117117481 shutdown_connections 2024-08-18T13:40:01.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 --2- 172.21.15.118:0/4117117481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a40c8410 0x7f79a40c2f70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f79a3fff700 1 -- 172.21.15.118:0/4117117481 >> 172.21.15.118:0/4117117481 conn(0x7f79a406c020 msgr2=0x7f79a40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ead9d700 1 -- 172.21.15.118:0/140165587 >> v2:172.21.15.178:3300/0 conn(0x7f96ec14b5d0 msgr2=0x7f96ec14b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:01.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ead9d700 1 --2- 172.21.15.118:0/140165587 >> v2:172.21.15.178:3300/0 conn(0x7f96ec14b5d0 0x7f96ec14b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ead9d700 1 -- 172.21.15.118:0/140165587 >> v1:172.21.15.178:6789/0 conn(0x7f96ec155510 legacy=0x7f96ec0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f79a3fff700 1 -- 172.21.15.118:0/4117117481 shutdown_connections 2024-08-18T13:40:01.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- 172.21.15.118:0/1336497333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 msgr2=0x7f72940673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 --2- 172.21.15.118:0/1336497333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f72940673d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f729406ca20 tx=0x7f7290009250 comp rx=0 tx=0).stop 2024-08-18T13:40:01.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f79a3fff700 1 -- 172.21.15.118:0/4117117481 wait complete. 2024-08-18T13:40:01.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.766+0000 7f96ead9d700 1 -- 172.21.15.118:0/140165587 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec06a540 con 0x7f96ec06ec60 2024-08-18T13:40:01.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96ead9d700 1 --2- 172.21.15.118:0/140165587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 0x7f96ec06f0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0c4690 tx=0x7f96e000d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602ebf9cb99223a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- 172.21.15.118:0/1695153401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc152370 msgr2=0x7faccc0c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 --2- 172.21.15.118:0/1695153401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc152370 0x7faccc0c1950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7facc00098e0 tx=0x7facc00091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- 172.21.15.118:0/1695153401 shutdown_connections 2024-08-18T13:40:01.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 --2- 172.21.15.118:0/1695153401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc152370 0x7faccc0c1950 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- 172.21.15.118:0/1695153401 >> 172.21.15.118:0/1695153401 conn(0x7faccc0c44b0 msgr2=0x7faccc0c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96e959a700 1 -- 172.21.15.118:0/140165587 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0012070 con 0x7f96ec06ec60 2024-08-18T13:40:01.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- 172.21.15.118:0/1336497333 shutdown_connections 2024-08-18T13:40:01.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 --2- 172.21.15.118:0/1336497333 >> v2:172.21.15.178:3300/0 conn(0x7f729406f490 0x7f7294155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 --2- 172.21.15.118:0/1336497333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f72940673d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- 172.21.15.118:0/1336497333 >> 172.21.15.118:0/1336497333 conn(0x7f729406c020 msgr2=0x7f72940c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96e959a700 1 -- 172.21.15.118:0/140165587 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e000d2e0 con 0x7f96ec06ec60 2024-08-18T13:40:01.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96e959a700 1 -- 172.21.15.118:0/140165587 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000f9b0 con 0x7f96ec06ec60 2024-08-18T13:40:01.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- 172.21.15.118:0/1695153401 shutdown_connections 2024-08-18T13:40:01.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- 172.21.15.118:0/1336497333 shutdown_connections 2024-08-18T13:40:01.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- 172.21.15.118:0/1336497333 wait complete. 2024-08-18T13:40:01.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- 172.21.15.118:0/1695153401 wait complete. 2024-08-18T13:40:01.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d66e1700 1 Processor -- start 2024-08-18T13:40:01.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d66e1700 1 -- start start 2024-08-18T13:40:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 -- 172.21.15.118:0/140165587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 msgr2=0x7f96ec06f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/140165587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 0x7f96ec06f0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0c4690 tx=0x7f96e000d630 comp rx=0 tx=0).stop 2024-08-18T13:40:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 -- 172.21.15.118:0/140165587 shutdown_connections 2024-08-18T13:40:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/140165587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 0x7f96ec06f0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/140165587 >> v2:172.21.15.178:3300/0 conn(0x7f96ec14b5d0 0x7f96ec14b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 -- 172.21.15.118:0/140165587 >> 172.21.15.118:0/140165587 conn(0x7f96ec0c8660 msgr2=0x7f96ec0c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 Processor -- start 2024-08-18T13:40:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f79a3fff700 1 Processor -- start 2024-08-18T13:40:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 Processor -- start 2024-08-18T13:40:01.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- start start 2024-08-18T13:40:01.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f79a3fff700 1 -- start start 2024-08-18T13:40:01.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- start start 2024-08-18T13:40:01.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d66e1700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0d014b5d0 0x7ff0d014b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d66e1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d006a310 con 0x7ff0d006db90 2024-08-18T13:40:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f729406f490 0x7f72940c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72940c2440 0x7f7294149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f729414a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7294147720 con 0x7f72940c7fc0 2024-08-18T13:40:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72941659c0 con 0x7f729406f490 2024-08-18T13:40:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729ae33700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7294169550 con 0x7f72940c2440 2024-08-18T13:40:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 -- 172.21.15.118:0/140165587 shutdown_connections 2024-08-18T13:40:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d66e1700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d006a800 con 0x7ff0d0155510 2024-08-18T13:40:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d66e1700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d00c3120 con 0x7ff0d014b5d0 2024-08-18T13:40:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d4c7e700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff0d006db90 0x7ff0d015bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45982/0 (socket says 172.21.15.118:45982) 2024-08-18T13:40:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0d4c7e700 1 -- 172.21.15.118:0/1104599844 learned_addr learned my addr 172.21.15.118:0/1104599844 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0cffff700 1 --2- 172.21.15.118:0/1104599844 >> v2:172.21.15.178:3300/0 conn(0x7ff0d014b5d0 0x7ff0d014b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f96f0cbe700 1 -- 172.21.15.118:0/140165587 wait complete. 2024-08-18T13:40:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f7299e31700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f729406f490 0x7f72940c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f7299e31700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f729406f490 0x7f72940c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:44118/0 (socket says 172.21.15.118:44118) 2024-08-18T13:40:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f7299e31700 1 -- 172.21.15.118:0/2343760278 learned_addr learned my addr 172.21.15.118:0/2343760278 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0c4fc0 0x7faccc0c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c57c0 0x7faccc147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc152370 0x7faccc1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc146a80 con 0x7faccc0c4fc0 2024-08-18T13:40:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc149e20 con 0x7faccc0c57c0 2024-08-18T13:40:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7facd14e7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc146540 con 0x7faccc152370 2024-08-18T13:40:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f729a632700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f729414a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7f7299630700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72940c2440 0x7f7294149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7faccaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0c4fc0 0x7faccc0c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7faccaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0c4fc0 0x7faccc0c35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45350/0 (socket says 172.21.15.118:45350) 2024-08-18T13:40:01.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7faccaffd700 1 -- 172.21.15.118:0/2478490026 learned_addr learned my addr 172.21.15.118:0/2478490026 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7faccb7fe700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc152370 0x7faccc1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79a40673d0 0x7f79a4153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0cffff700 1 -- 172.21.15.118:0/1104599844 >> v1:172.21.15.178:6789/0 conn(0x7ff0d0155510 legacy=0x7ff0d00673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0cffff700 1 -- 172.21.15.118:0/1104599844 >> v1:172.21.15.118:6789/0 conn(0x7ff0d006db90 legacy=0x7ff0d015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.770+0000 7ff0cffff700 1 -- 172.21.15.118:0/1104599844 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d006a540 con 0x7ff0d014b5d0 2024-08-18T13:40:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a3fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79a40c8410 0x7f79a41496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79a40673d0 0x7f79a4153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79a40673d0 0x7f79a4153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45366/0 (socket says 172.21.15.118:45366) 2024-08-18T13:40:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 -- 172.21.15.118:0/212234835 learned_addr learned my addr 172.21.15.118:0/212234835 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a3fff700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a414b630 0x7f79a4166860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7faccaffd700 1 -- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc152370 msgr2=0x7faccc1498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7faccaffd700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc152370 0x7faccc1498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7faccaffd700 1 -- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c57c0 msgr2=0x7faccc147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7facca7fc700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c57c0 0x7faccc147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0cffff700 1 --2- 172.21.15.118:0/1104599844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d014b5d0 0x7ff0d014b9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800f300 tx=0x7ff0c800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b528611009a2b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7299e31700 1 -- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72940c2440 msgr2=0x7f7294149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7299e31700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72940c2440 0x7f7294149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7299e31700 1 -- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 msgr2=0x7f729414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41641c0 con 0x7f79a40673d0 2024-08-18T13:40:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7faccaffd700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c57c0 0x7faccc147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7faccaffd700 1 -- 172.21.15.118:0/2478490026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0009420 con 0x7faccc0c4fc0 2024-08-18T13:40:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4163d30 con 0x7f79a414b630 2024-08-18T13:40:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7299e31700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f729414a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7299e31700 1 -- 172.21.15.118:0/2343760278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900094f0 con 0x7f729406f490 2024-08-18T13:40:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4154830 con 0x7f79a40c8410 2024-08-18T13:40:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7facca7fc700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c57c0 0x7faccc147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:40:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f729a632700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f729414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:01.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7faccaffd700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0c4fc0 0x7faccc0c35a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faccc06abc0 tx=0x7facc0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0ceffd700 1 -- 172.21.15.118:0/1104599844 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8012070 con 0x7ff0d014b5d0 2024-08-18T13:40:01.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a37fe700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a414b630 0x7f79a4166860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 Processor -- start 2024-08-18T13:40:01.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a27fc700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79a40c8410 0x7f79a41496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7facb2ffd700 1 -- 172.21.15.118:0/2478490026 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000bbe0 con 0x7faccc0c4fc0 2024-08-18T13:40:01.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7facb2ffd700 1 -- 172.21.15.118:0/2478490026 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0011420 con 0x7faccc0c4fc0 2024-08-18T13:40:01.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0ceffd700 1 -- 172.21.15.118:0/1104599844 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c800d0d0 con 0x7ff0d014b5d0 2024-08-18T13:40:01.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0ceffd700 1 -- 172.21.15.118:0/1104599844 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800d490 con 0x7ff0d014b5d0 2024-08-18T13:40:01.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7299e31700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f729406f490 0x7f72940c64a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f728400c590 tx=0x7f728400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 -- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79a40c8410 msgr2=0x7f79a41496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79a40c8410 0x7f79a41496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 -- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a414b630 msgr2=0x7f79a4166860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a414b630 0x7f79a4166860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799400a5d0 con 0x7f79a40673d0 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a2ffd700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79a40673d0 0x7f79a4153d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c900 tx=0x7f799c002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f798affd700 1 -- 172.21.15.118:0/212234835 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c012630 con 0x7f79a40673d0 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f798affd700 1 -- 172.21.15.118:0/212234835 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c012790 con 0x7f79a40673d0 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f798affd700 1 -- 172.21.15.118:0/212234835 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c012b50 con 0x7f79a40673d0 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0d66e1700 1 -- 172.21.15.118:0/1104599844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d014b5d0 msgr2=0x7ff0d014b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/1104599844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d014b5d0 0x7ff0d014b9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800f300 tx=0x7ff0c800db90 comp rx=0 tx=0).stop 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a4163440 con 0x7f79a40673d0 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37b5128700 1 Processor -- start 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 -- start start 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 0x7f96ec14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96ec14b5d0 0x7f96ec16bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec155510 0x7f96ec16c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0bf9b0 con 0x7f96ec06ec60 2024-08-18T13:40:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec14da60 con 0x7f96ec155510 2024-08-18T13:40:01.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96f0cbe700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec16ae50 con 0x7f96ec14b5d0 2024-08-18T13:40:01.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37b5128700 1 -- start start 2024-08-18T13:40:01.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7289ffb700 1 -- 172.21.15.118:0/2343760278 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284011590 con 0x7f729406f490 2024-08-18T13:40:01.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7289ffb700 1 -- 172.21.15.118:0/2343760278 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284011710 con 0x7f729406f490 2024-08-18T13:40:01.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f7289ffb700 1 -- 172.21.15.118:0/2343760278 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284011ad0 con 0x7f729406f490 2024-08-18T13:40:01.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0d66e1700 1 -- 172.21.15.118:0/1104599844 shutdown_connections 2024-08-18T13:40:01.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/1104599844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d014b5d0 0x7ff0d014b9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0d66e1700 1 -- 172.21.15.118:0/1104599844 >> 172.21.15.118:0/1104599844 conn(0x7ff0d00c8660 msgr2=0x7ff0d00c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0d66e1700 1 -- 172.21.15.118:0/1104599844 shutdown_connections 2024-08-18T13:40:01.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37b5128700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37b014b520 0x7f37b0155400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7ff0d66e1700 1 -- 172.21.15.118:0/1104599844 wait complete. 2024-08-18T13:40:01.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7facb2ffd700 1 -- 172.21.15.118:0/2478490026 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00117e0 con 0x7faccc0c4fc0 2024-08-18T13:40:01.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc1467e0 con 0x7faccc0c4fc0 2024-08-18T13:40:01.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc0c2da0 con 0x7faccc0c4fc0 2024-08-18T13:40:01.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37b5128700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00c6600 con 0x7f37b0067ae0 2024-08-18T13:40:01.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37b5128700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00c7460 con 0x7f37b00c57c0 2024-08-18T13:40:01.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37b5128700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b00c6850 con 0x7f37b014b520 2024-08-18T13:40:01.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37aed9d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37b014b520 0x7f37b0155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37aed9d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37b014b520 0x7f37b0155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:44144/0 (socket says 172.21.15.118:44144) 2024-08-18T13:40:01.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37aed9d700 1 -- 172.21.15.118:0/2371494101 learned_addr learned my addr 172.21.15.118:0/2371494101 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96ead9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec155510 0x7f96ec16c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96ead9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec155510 0x7f96ec16c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:44150/0 (socket says 172.21.15.118:44150) 2024-08-18T13:40:01.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f96ead9d700 1 -- 172.21.15.118:0/1605811728 learned_addr learned my addr 172.21.15.118:0/1605811728 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37aed9d700 1 -- 172.21.15.118:0/2371494101 >> v1:172.21.15.178:6789/0 conn(0x7f37b00c57c0 legacy=0x7f37b00673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37aed9d700 1 -- 172.21.15.118:0/2371494101 >> v1:172.21.15.118:6789/0 conn(0x7f37b0067ae0 legacy=0x7f37b00c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f37aed9d700 1 -- 172.21.15.118:0/2371494101 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00c6b30 con 0x7f37b014b520 2024-08-18T13:40:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f96ea59c700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 0x7f96ec14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f96ead9d700 1 -- 172.21.15.118:0/1605811728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96ec14b5d0 msgr2=0x7f96ec16bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f96ead9d700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96ec14b5d0 0x7f96ec16bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f96ead9d700 1 -- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 msgr2=0x7f96ec14cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f96ead9d700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 0x7f96ec14cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f96ead9d700 1 -- 172.21.15.118:0/1605811728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e000d8d0 con 0x7f96ec155510 2024-08-18T13:40:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f37aed9d700 1 --2- 172.21.15.118:0/2371494101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b014b520 0x7f37b0155400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37a40090e0 tx=0x7f37a4009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9763f799ec3657 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f96ead9d700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec155510 0x7f96ec16c080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96e000d670 tx=0x7f96e001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 Processor -- start 2024-08-18T13:40:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72940c5750 con 0x7f729406f490 2024-08-18T13:40:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.774+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7294165370 con 0x7f729406f490 2024-08-18T13:40:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72941651d0 con 0x7f729406f490 2024-08-18T13:40:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 -- start start 2024-08-18T13:40:01.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7facb2ffd700 1 -- 172.21.15.118:0/2478490026 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7facc0011c10 con 0x7faccc0c4fc0 2024-08-18T13:40:01.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7facb2ffd700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb80608e0 0x7facb8062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7facb2ffd700 1 -- 172.21.15.118:0/2478490026 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7facc007e8c0 con 0x7faccc0c4fc0 2024-08-18T13:40:01.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7facca7fc700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb80608e0 0x7facb8062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f798affd700 1 -- 172.21.15.118:0/212234835 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f799c01b400 con 0x7f79a40673d0 2024-08-18T13:40:01.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41636e0 con 0x7f79a40673d0 2024-08-18T13:40:01.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f7289ffb700 1 -- 172.21.15.118:0/2343760278 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7284033ad0 con 0x7f729406f490 2024-08-18T13:40:01.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7facca7fc700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb80608e0 0x7facb8062d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7facc4007fd0 tx=0x7facc4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f7289ffb700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7278060930 0x7f7278062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f798affd700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7990060890 0x7f7990062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.813 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f7289ffb700 1 -- 172.21.15.118:0/2343760278 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f728407c070 con 0x7f729406f490 2024-08-18T13:40:01.813 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f7299630700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7278060930 0x7f7278062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.813 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f798affd700 1 -- 172.21.15.118:0/212234835 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f799c07c510 con 0x7f79a40673d0 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d006db90 0x7ff0d0153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0d014b5d0 0x7ff0d013e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d0155510 0x7ff0d013ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01414b0 con 0x7ff0d014b5d0 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d013ff50 con 0x7ff0d006db90 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d014c5b0 con 0x7ff0d0155510 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d0155510 0x7ff0d013ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d0155510 0x7ff0d013ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54498/0 (socket says 172.21.15.118:54498) 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 -- 172.21.15.118:0/572515100 learned_addr learned my addr 172.21.15.118:0/572515100 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0cf7fe700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0d014b5d0 0x7ff0d013e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 -- 172.21.15.118:0/572515100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d006db90 msgr2=0x7ff0d0153aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T13:40:01.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d006db90 0x7ff0d0153aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 -- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0d014b5d0 msgr2=0x7ff0d013e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0d014b5d0 0x7ff0d013e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 -- 172.21.15.118:0/572515100 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c800de30 con 0x7ff0d0155510 2024-08-18T13:40:01.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0cf7fe700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0d014b5d0 0x7ff0d013e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:40:01.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f7299630700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7278060930 0x7f7278062de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f729406ca20 tx=0x7f7290017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f79a27fc700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7990060890 0x7f7990062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a4071410 con 0x7f79a40673d0 2024-08-18T13:40:01.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d4c7e700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d0155510 0x7ff0d013ea40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000f60 tx=0x7ff0c4007e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7f79a27fc700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7990060890 0x7f7990062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7998004e00 tx=0x7f7998000bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0b7fff700 1 -- 172.21.15.118:0/572515100 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4005b30 con 0x7ff0d0155510 2024-08-18T13:40:01.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0b7fff700 1 -- 172.21.15.118:0/572515100 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4005cb0 con 0x7ff0d0155510 2024-08-18T13:40:01.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d0161660 con 0x7ff0d0155510 2024-08-18T13:40:01.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0b7fff700 1 -- 172.21.15.118:0/572515100 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007340 con 0x7ff0d0155510 2024-08-18T13:40:01.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.778+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d013f900 con 0x7ff0d0155510 2024-08-18T13:40:01.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37add9b700 1 -- 172.21.15.118:0/2371494101 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a400f750 con 0x7f37b014b520 2024-08-18T13:40:01.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafe064700 1 Processor -- start 2024-08-18T13:40:01.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37add9b700 1 -- 172.21.15.118:0/2371494101 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a400f8b0 con 0x7f37b014b520 2024-08-18T13:40:01.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37add9b700 1 -- 172.21.15.118:0/2371494101 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a400fc70 con 0x7f37b014b520 2024-08-18T13:40:01.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafe064700 1 -- start start 2024-08-18T13:40:01.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- 172.21.15.118:0/2371494101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b014b520 msgr2=0x7f37b0155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 --2- 172.21.15.118:0/2371494101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b014b520 0x7f37b0155400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37a40090e0 tx=0x7f37a4009ae0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc071480 con 0x7faccc0c4fc0 2024-08-18T13:40:01.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- 172.21.15.118:0/2371494101 shutdown_connections 2024-08-18T13:40:01.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 --2- 172.21.15.118:0/2371494101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b014b520 0x7f37b0155400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.820 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- 172.21.15.118:0/2371494101 >> 172.21.15.118:0/2371494101 conn(0x7f37b00c44b0 msgr2=0x7f37b00c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.820 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- 172.21.15.118:0/2371494101 shutdown_connections 2024-08-18T13:40:01.820 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- 172.21.15.118:0/2371494101 wait complete. 2024-08-18T13:40:01.820 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafe064700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcaf8155100 0x7fcaf80c2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafe064700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcaf80c3de0 0x7fcaf8068fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcaf80c3de0 0x7fcaf8068fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcaf80c3de0 0x7fcaf8068fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45400/0 (socket says 172.21.15.118:45400) 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 -- 172.21.15.118:0/3741448967 learned_addr learned my addr 172.21.15.118:0/3741448967 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf8068b30 con 0x7fcaf806ded0 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf806f370 con 0x7fcaf80c3de0 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf806e9b0 con 0x7fcaf8155100 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafc861700 1 --2- 172.21.15.118:0/3741448967 >> v2:172.21.15.178:3300/0 conn(0x7fcaf8155100 0x7fcaf80c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 -- 172.21.15.118:0/3741448967 >> v2:172.21.15.178:3300/0 conn(0x7fcaf8155100 msgr2=0x7fcaf80c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 --2- 172.21.15.118:0/3741448967 >> v2:172.21.15.178:3300/0 conn(0x7fcaf8155100 0x7fcaf80c2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 -- 172.21.15.118:0/3741448967 >> v1:172.21.15.178:6789/0 conn(0x7fcaf806ded0 legacy=0x7fcaf806e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 -- 172.21.15.118:0/3741448967 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8067f20 con 0x7fcaf80c3de0 2024-08-18T13:40:01.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7ff0b7fff700 1 -- 172.21.15.118:0/572515100 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff0c401b400 con 0x7ff0d0155510 2024-08-18T13:40:01.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7fcafd863700 1 --2- 172.21.15.118:0/3741448967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 0x7fcaf8068fc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcaf806d4a0 tx=0x7fcaf0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3079df9460a45373 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7ff0b7fff700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff0bc060860 0x7ff0bc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7ff0b7fff700 1 -- 172.21.15.118:0/572515100 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7ff0c407cab0 con 0x7ff0d0155510 2024-08-18T13:40:01.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 Processor -- start 2024-08-18T13:40:01.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- start start 2024-08-18T13:40:01.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7ff0cffff700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff0bc060860 0x7ff0bc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7ff0cffff700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff0bc060860 0x7ff0bc062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800f300 tx=0x7ff0c801b000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b0067ae0 0x7f37b014c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b00c57c0 0x7f37b014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b014b520 0x7f37b014de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b006f710 con 0x7f37b014b520 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b014d4b0 con 0x7f37b0067ae0 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37b5128700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0165bf0 con 0x7f37b00c57c0 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b014b520 0x7f37b014de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b014b520 0x7f37b014de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45408/0 (socket says 172.21.15.118:45408) 2024-08-18T13:40:01.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 -- 172.21.15.118:0/1484450562 learned_addr learned my addr 172.21.15.118:0/1484450562 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37aed9d700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b0067ae0 0x7f37b014c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37ae59c700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b00c57c0 0x7f37b014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 -- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b00c57c0 msgr2=0x7f37b014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b00c57c0 0x7f37b014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 -- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b0067ae0 msgr2=0x7f37b014c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b0067ae0 0x7f37b014c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.782+0000 7f37af59e700 1 -- 172.21.15.118:0/1484450562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009d80 con 0x7f37b014b520 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f37af59e700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b014b520 0x7f37b014de80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b610 tx=0x7f37a0009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d00642f0 con 0x7ff0d0155510 2024-08-18T13:40:01.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f798affd700 1 -- 172.21.15.118:0/212234835 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f799c081060 con 0x7f79a40673d0 2024-08-18T13:40:01.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f96da7fc700 1 -- 172.21.15.118:0/1605811728 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000b070 con 0x7f96ec155510 2024-08-18T13:40:01.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f96da7fc700 1 -- 172.21.15.118:0/1605811728 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0016ce0 con 0x7f96ec155510 2024-08-18T13:40:01.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec14bf10 con 0x7f96ec155510 2024-08-18T13:40:01.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec16b060 con 0x7f96ec155510 2024-08-18T13:40:01.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f96da7fc700 1 -- 172.21.15.118:0/1605811728 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e001e640 con 0x7f96ec155510 2024-08-18T13:40:01.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f7289ffb700 1 -- 172.21.15.118:0/2343760278 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f728404c330 con 0x7f729406f490 2024-08-18T13:40:01.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d00052b0 con 0x7f96ec155510 2024-08-18T13:40:01.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.786+0000 7f96da7fc700 1 -- 172.21.15.118:0/1605811728 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f96e001e9f0 con 0x7f96ec155510 2024-08-18T13:40:01.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.790+0000 7f96da7fc700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96cc060860 0x7f96cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.790+0000 7f96da7fc700 1 -- 172.21.15.118:0/1605811728 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f96e007d4e0 con 0x7f96ec155510 2024-08-18T13:40:01.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.790+0000 7f96ea59c700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96cc060860 0x7f96cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.790+0000 7f96ea59c700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96cc060860 0x7f96cc062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96e400b570 tx=0x7f96e400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.790+0000 7f3796ffd700 1 -- 172.21.15.118:0/1484450562 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a000f510 con 0x7f37b014b520 2024-08-18T13:40:01.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.790+0000 7fcaeffff700 1 -- 172.21.15.118:0/3741448967 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000e070 con 0x7fcaf80c3de0 2024-08-18T13:40:01.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7f3796ffd700 1 -- 172.21.15.118:0/1484450562 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a000f670 con 0x7f37b014b520 2024-08-18T13:40:01.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7fcaeffff700 1 -- 172.21.15.118:0/3741448967 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf000ba20 con 0x7fcaf80c3de0 2024-08-18T13:40:01.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7facb2ffd700 1 -- 172.21.15.118:0/2478490026 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7facc004e980 con 0x7faccc0c4fc0 2024-08-18T13:40:01.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7f3796ffd700 1 -- 172.21.15.118:0/1484450562 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a000fa30 con 0x7f37b014b520 2024-08-18T13:40:01.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7f37b5128700 1 -- 172.21.15.118:0/1484450562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b0165ed0 con 0x7f37b014b520 2024-08-18T13:40:01.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7f37b5128700 1 -- 172.21.15.118:0/1484450562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b0166490 con 0x7f37b014b520 2024-08-18T13:40:01.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 msgr2=0x7fcaf8068fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7fcafe064700 1 --2- 172.21.15.118:0/3741448967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 0x7fcaf8068fc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcaf806d4a0 tx=0x7fcaf0009840 comp rx=0 tx=0).stop 2024-08-18T13:40:01.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7ff0b7fff700 1 -- 172.21.15.118:0/572515100 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff0c404ccc0 con 0x7ff0d0155510 2024-08-18T13:40:01.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.794+0000 7f96da7fc700 1 -- 172.21.15.118:0/1605811728 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f96e004d6f0 con 0x7f96ec155510 2024-08-18T13:40:01.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.798+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 shutdown_connections 2024-08-18T13:40:01.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.798+0000 7fcafe064700 1 --2- 172.21.15.118:0/3741448967 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 0x7fcaf8068fc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.798+0000 7fcafe064700 1 --2- 172.21.15.118:0/3741448967 >> v2:172.21.15.178:3300/0 conn(0x7fcaf8155100 0x7fcaf80c2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.798+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 >> 172.21.15.118:0/3741448967 conn(0x7fcaf80c4230 msgr2=0x7fcaf80c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:01.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7f3796ffd700 1 -- 172.21.15.118:0/1484450562 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f37a001a400 con 0x7f37b014b520 2024-08-18T13:40:01.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7f3796ffd700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f379c060860 0x7f379c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7f3796ffd700 1 -- 172.21.15.118:0/1484450562 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f37a007cb40 con 0x7f37b014b520 2024-08-18T13:40:01.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7f37aed9d700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f379c060860 0x7f379c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 shutdown_connections 2024-08-18T13:40:01.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7f37aed9d700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f379c060860 0x7f379c062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f37a40090e0 tx=0x7f37a4018000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 -- 172.21.15.118:0/3741448967 wait complete. 2024-08-18T13:40:01.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 Processor -- start 2024-08-18T13:40:01.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b014d280 con 0x7f37b014b520 2024-08-18T13:40:01.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 -- start start 2024-08-18T13:40:01.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf806ded0 0x7fcaf816ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 0x7fcaf8151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf8155100 0x7fcaf80c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf816c970 con 0x7fcaf80c3de0 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81524b0 con 0x7fcaf806ded0 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafe064700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8169e50 con 0x7fcaf8155100 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafd062700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf806ded0 0x7fcaf816ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafd863700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf8155100 0x7fcaf80c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafd863700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf8155100 0x7fcaf80c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54516/0 (socket says 172.21.15.118:54516) 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafd863700 1 -- 172.21.15.118:0/1830148722 learned_addr learned my addr 172.21.15.118:0/1830148722 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafd062700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf806ded0 0x7fcaf816ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:44178/0 (socket says 172.21.15.118:44178) 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.802+0000 7fcafc861700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 0x7fcaf8151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafd062700 1 -- 172.21.15.118:0/1830148722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf8155100 msgr2=0x7fcaf80c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafd062700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf8155100 0x7fcaf80c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafd062700 1 -- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 msgr2=0x7fcaf8151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafd062700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 0x7fcaf8151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafd062700 1 -- 172.21.15.118:0/1830148722 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0009ae0 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafd062700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf806ded0 0x7fcaf816ce00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c490 tx=0x7fcaf4007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcaed7fa700 1 -- 172.21.15.118:0/1830148722 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400ce40 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf8152730 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.806+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf816c600 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.810+0000 7fcaed7fa700 1 -- 172.21.15.118:0/1830148722 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4011590 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.810+0000 7fcaed7fa700 1 -- 172.21.15.118:0/1830148722 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4011950 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.810+0000 7fcaed7fa700 1 -- 172.21.15.118:0/1830148722 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcaf4011cc0 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.810+0000 7fcaed7fa700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcae4060b60 0x7fcae4063010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.810+0000 7fcaed7fa700 1 -- 172.21.15.118:0/1830148722 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fcaf407c030 con 0x7fcaf806ded0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.810+0000 7fcafc861700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcae4060b60 0x7fcae4063010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.814+0000 7fcafc861700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcae4060b60 0x7fcae4063010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80be6b0 tx=0x7fcae80091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:01.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.814+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc0052b0 con 0x7fcaf806ded0 2024-08-18T13:40:01.838 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.814+0000 7f3796ffd700 1 -- 172.21.15.118:0/1484450562 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f37b014d280 con 0x7f37b014b520 2024-08-18T13:40:01.838 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:01.822+0000 7fcaed7fa700 1 -- 172.21.15.118:0/1830148722 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcaf404a480 con 0x7fcaf806ded0 2024-08-18T13:40:02.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.054+0000 7f9ee3a36700 1 -- 172.21.15.118:0/1008516067 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9edc14ae90 con 0x7f9edc0c57c0 2024-08-18T13:40:02.057 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.054+0000 7f9ed17fa700 1 -- 172.21.15.118:0/1008516067 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ed4052ca0 con 0x7f9edc0c57c0 2024-08-18T13:40:02.060 INFO:teuthology.orchestra.run.smithi118.stdout:60129542154 2024-08-18T13:40:02.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 -- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9ec0060930 msgr2=0x7f9ec0062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9ec0060930 0x7f9ec0062de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9edc164ca0 tx=0x7f9ecc008040 comp rx=0 tx=0).stop 2024-08-18T13:40:02.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 -- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9edc0c57c0 msgr2=0x7f9edc165000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9edc0c57c0 0x7f9edc165000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0c6990 tx=0x7f9ed4017000 comp rx=0 tx=0).stop 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 -- 172.21.15.118:0/1008516067 shutdown_connections 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9ec0060930 0x7f9ec0062de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9edc163cb0 0x7f9edc175910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9edc14b8f0 0x7f9edc163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 --2- 172.21.15.118:0/1008516067 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9edc0c57c0 0x7f9edc165000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 -- 172.21.15.118:0/1008516067 >> 172.21.15.118:0/1008516067 conn(0x7f9edc0c44b0 msgr2=0x7f9edc06d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 -- 172.21.15.118:0/1008516067 shutdown_connections 2024-08-18T13:40:02.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.062+0000 7f9ec6ffd700 1 -- 172.21.15.118:0/1008516067 wait complete. 2024-08-18T13:40:02.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.0 2024-08-18T13:40:02.080 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.082+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faccc0c5cb0 con 0x7faccc0c4fc0 2024-08-18T13:40:02.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facb2ffd700 1 -- 172.21.15.118:0/2478490026 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7facc0015020 con 0x7faccc0c4fc0 2024-08-18T13:40:02.089 INFO:teuthology.orchestra.run.smithi118.stdout:64424509452 2024-08-18T13:40:02.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f72941651d0 con 0x7f729406f490 2024-08-18T13:40:02.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7f7289ffb700 1 -- 172.21.15.118:0/2343760278 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7284017020 con 0x7f729406f490 2024-08-18T13:40:02.091 INFO:teuthology.orchestra.run.smithi118.stdout:60129542154 2024-08-18T13:40:02.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f96d0005830 con 0x7f96ec155510 2024-08-18T13:40:02.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb80608e0 msgr2=0x7facb8062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb80608e0 0x7facb8062d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7facc4007fd0 tx=0x7facc4009040 comp rx=0 tx=0).stop 2024-08-18T13:40:02.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0c4fc0 msgr2=0x7faccc0c35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0c4fc0 0x7faccc0c35a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faccc06abc0 tx=0x7facc0017000 comp rx=0 tx=0).stop 2024-08-18T13:40:02.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 shutdown_connections 2024-08-18T13:40:02.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb80608e0 0x7facb8062d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc152370 0x7faccc1498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c57c0 0x7faccc147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 --2- 172.21.15.118:0/2478490026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0c4fc0 0x7faccc0c35a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 >> 172.21.15.118:0/2478490026 conn(0x7faccc0c44b0 msgr2=0x7faccc0c5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 shutdown_connections 2024-08-18T13:40:02.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7facd14e7700 1 -- 172.21.15.118:0/2478490026 wait complete. 2024-08-18T13:40:02.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.086+0000 7f96da7fc700 1 -- 172.21.15.118:0/1605811728 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f96e0050d10 con 0x7f96ec155510 2024-08-18T13:40:02.095 INFO:teuthology.orchestra.run.smithi118.stdout:60129542154 2024-08-18T13:40:02.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f79a406c5c0 con 0x7f79a40673d0 2024-08-18T13:40:02.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f798affd700 1 -- 172.21.15.118:0/212234835 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f799c016030 con 0x7f79a40673d0 2024-08-18T13:40:02.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96cc060860 msgr2=0x7f96cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96cc060860 0x7f96cc062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96e400b570 tx=0x7f96e400b040 comp rx=0 tx=0).stop 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec155510 msgr2=0x7f96ec16c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec155510 0x7f96ec16c080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96e000d670 tx=0x7f96e001c000 comp rx=0 tx=0).stop 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 shutdown_connections 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96cc060860 0x7f96cc062d10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96ec155510 0x7f96ec16c080 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96ec14b5d0 0x7f96ec16bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 --2- 172.21.15.118:0/1605811728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96ec06ec60 0x7f96ec14cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 >> 172.21.15.118:0/1605811728 conn(0x7f96ec0c8660 msgr2=0x7f96ec0c73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 shutdown_connections 2024-08-18T13:40:02.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f96f0cbe700 1 -- 172.21.15.118:0/1605811728 wait complete. 2024-08-18T13:40:02.098 INFO:teuthology.orchestra.run.smithi118.stdout:64424509451 2024-08-18T13:40:02.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7278060930 msgr2=0x7f7278062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7278060930 0x7f7278062de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f729406ca20 tx=0x7f7290017000 comp rx=0 tx=0).stop 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f729406f490 msgr2=0x7f72940c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f729406f490 0x7f72940c64a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f728400c590 tx=0x7f728400c8a0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 shutdown_connections 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7278060930 0x7f7278062de0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72940c7fc0 0x7f729414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72940c2440 0x7f7294149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 --2- 172.21.15.118:0/2343760278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f729406f490 0x7f72940c64a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 >> 172.21.15.118:0/2343760278 conn(0x7f729406c020 msgr2=0x7f72940bfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 shutdown_connections 2024-08-18T13:40:02.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.090+0000 7f729ae33700 1 -- 172.21.15.118:0/2343760278 wait complete. 2024-08-18T13:40:02.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.094+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7990060890 msgr2=0x7f7990062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.094+0000 7f79a3fff700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7990060890 0x7f7990062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7998004e00 tx=0x7f7998000bc0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.094+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79a40673d0 msgr2=0x7f79a4153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.094+0000 7f79a3fff700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79a40673d0 0x7f79a4153d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c900 tx=0x7f799c002780 comp rx=0 tx=0).stop 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 shutdown_connections 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7990060890 0x7f7990062d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79a414b630 0x7f79a4166860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79a40c8410 0x7f79a41496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 --2- 172.21.15.118:0/212234835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79a40673d0 0x7f79a4153d10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 >> 172.21.15.118:0/212234835 conn(0x7f79a406c020 msgr2=0x7f79a40c7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 shutdown_connections 2024-08-18T13:40:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7f79a3fff700 1 -- 172.21.15.118:0/212234835 wait complete. 2024-08-18T13:40:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff0d006e120 con 0x7ff0d0155510 2024-08-18T13:40:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.098+0000 7ff0b7fff700 1 -- 172.21.15.118:0/572515100 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0c40502e0 con 0x7ff0d0155510 2024-08-18T13:40:02.104 INFO:teuthology.orchestra.run.smithi118.stdout:55834574858 2024-08-18T13:40:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff0bc060860 msgr2=0x7ff0bc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff0bc060860 0x7ff0bc062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800f300 tx=0x7ff0c801b000 comp rx=0 tx=0).stop 2024-08-18T13:40:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d0155510 msgr2=0x7ff0d013ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d0155510 0x7ff0d013ea40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000f60 tx=0x7ff0c4007e70 comp rx=0 tx=0).stop 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 shutdown_connections 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff0bc060860 0x7ff0bc062d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0d0155510 0x7ff0d013ea40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0d014b5d0 0x7ff0d013e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 --2- 172.21.15.118:0/572515100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0d006db90 0x7ff0d0153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 >> 172.21.15.118:0/572515100 conn(0x7ff0d00c8660 msgr2=0x7ff0d0067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 shutdown_connections 2024-08-18T13:40:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.102+0000 7ff0d66e1700 1 -- 172.21.15.118:0/572515100 wait complete. 2024-08-18T13:40:02.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.1 2024-08-18T13:40:02.110 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.6 2024-08-18T13:40:02.111 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.2 2024-08-18T13:40:02.112 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.5 2024-08-18T13:40:02.119 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.7 2024-08-18T13:40:02.119 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.120 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.114+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f37b014d280 con 0x7f37b014b520 2024-08-18T13:40:02.121 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.114+0000 7f3796ffd700 1 -- 172.21.15.118:0/1484450562 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37b014d280 con 0x7f37b014b520 2024-08-18T13:40:02.121 INFO:teuthology.orchestra.run.smithi118.stdout:64424509450 2024-08-18T13:40:02.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcadc005560 con 0x7fcaf806ded0 2024-08-18T13:40:02.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f379c060860 msgr2=0x7f379c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f379c060860 0x7f379c062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f37a40090e0 tx=0x7f37a4018000 comp rx=0 tx=0).stop 2024-08-18T13:40:02.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b014b520 msgr2=0x7f37b014de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b014b520 0x7f37b014de80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b610 tx=0x7f37a0009060 comp rx=0 tx=0).stop 2024-08-18T13:40:02.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 shutdown_connections 2024-08-18T13:40:02.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f379c060860 0x7f379c062d10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b014b520 0x7f37b014de80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b00c57c0 0x7f37b014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 --2- 172.21.15.118:0/1484450562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b0067ae0 0x7f37b014c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 >> 172.21.15.118:0/1484450562 conn(0x7f37b00c44b0 msgr2=0x7f37b0068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7fcaed7fa700 1 -- 172.21.15.118:0/1830148722 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcaf4017020 con 0x7fcaf806ded0 2024-08-18T13:40:02.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 shutdown_connections 2024-08-18T13:40:02.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.118+0000 7f3794ff9700 1 -- 172.21.15.118:0/1484450562 wait complete. 2024-08-18T13:40:02.127 INFO:teuthology.orchestra.run.smithi118.stdout:60129542155 2024-08-18T13:40:02.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcae4060b60 msgr2=0x7fcae4063010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcae4060b60 0x7fcae4063010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80be6b0 tx=0x7fcae80091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf806ded0 msgr2=0x7fcaf816ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf806ded0 0x7fcaf816ce00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c490 tx=0x7fcaf4007ec0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 shutdown_connections 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcae4060b60 0x7fcae4063010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcaf8155100 0x7fcaf80c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcaf80c3de0 0x7fcaf8151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 --2- 172.21.15.118:0/1830148722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcaf806ded0 0x7fcaf816ce00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 >> 172.21.15.118:0/1830148722 conn(0x7fcaf80c4230 msgr2=0x7fcaf80c0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 shutdown_connections 2024-08-18T13:40:02.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.122+0000 7fcafe064700 1 -- 172.21.15.118:0/1830148722 wait complete. 2024-08-18T13:40:02.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.4 2024-08-18T13:40:02.137 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.3 2024-08-18T13:40:02.139 DEBUG:teuthology.parallel:result is None 2024-08-18T13:40:02.139 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-18T13:40:02.139 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:40:02.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85f4fd700 1 Processor -- start 2024-08-18T13:40:02.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85f4fd700 1 -- start start 2024-08-18T13:40:02.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85f4fd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85f4fd700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa858067470 con 0x7fa8580c0c50 2024-08-18T13:40:02.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85f4fd700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa858067710 con 0x7fa85814bf60 2024-08-18T13:40:02.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85f4fd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa858068370 con 0x7fa85805e6b0 2024-08-18T13:40:02.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85dcfa700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85e4fb700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa85814bf60 0x7fa85806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:34120/0 (socket says 172.21.15.118:34120) 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85e4fb700 1 -- 172.21.15.118:0/2103958662 learned_addr learned my addr 172.21.15.118:0/2103958662 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85ecfc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fa8580c0c50 0x7fa8580c10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:45996/0 (socket says 172.21.15.118:45996) 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85d4f9700 1 -- 172.21.15.118:0/2103958662 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171717607 0 0) 0x7fa858067470 con 0x7fa8580c0c50 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85d4f9700 1 -- 172.21.15.118:0/2103958662 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa844003570 con 0x7fa8580c0c50 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.278+0000 7fa85dcfa700 1 -- 172.21.15.118:0/2103958662 >> v1:172.21.15.178:6789/0 conn(0x7fa85814bf60 legacy=0x7fa85806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85dcfa700 1 -- 172.21.15.118:0/2103958662 >> v1:172.21.15.118:6789/0 conn(0x7fa8580c0c50 legacy=0x7fa8580c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85dcfa700 1 -- 172.21.15.118:0/2103958662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85806cb10 con 0x7fa85805e6b0 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85dcfa700 1 --2- 172.21.15.118:0/2103958662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858152840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d100 tx=0x7fa85000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189d3865114e5718 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85d4f9700 1 -- 172.21.15.118:0/2103958662 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850010070 con 0x7fa85805e6b0 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85d4f9700 1 -- 172.21.15.118:0/2103958662 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85000de00 con 0x7fa85805e6b0 2024-08-18T13:40:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85d4f9700 1 -- 172.21.15.118:0/2103958662 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000b2e0 con 0x7fa85805e6b0 2024-08-18T13:40:02.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2103958662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 msgr2=0x7fa858152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2103958662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858152840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d100 tx=0x7fa85000b990 comp rx=0 tx=0).stop 2024-08-18T13:40:02.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2103958662 shutdown_connections 2024-08-18T13:40:02.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2103958662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858152840 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2103958662 >> 172.21.15.118:0/2103958662 conn(0x7fa8580c81e0 msgr2=0x7fa8580c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2103958662 shutdown_connections 2024-08-18T13:40:02.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2103958662 wait complete. 2024-08-18T13:40:02.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 Processor -- start 2024-08-18T13:40:02.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- start start 2024-08-18T13:40:02.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858165bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8580c0c50 0x7fa858168fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa85814bf60 0x7fa85806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa858152d80 con 0x7fa85814bf60 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85814e430 con 0x7fa8580c0c50 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.282+0000 7fa85f4fd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa858157ce0 con 0x7fa85805e6b0 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa85814bf60 0x7fa85806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85dcfa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8580c0c50 0x7fa858168fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85e4fb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858165bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa85814bf60 0x7fa85806a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45430/0 (socket says 172.21.15.118:45430) 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 -- 172.21.15.118:0/2902240783 learned_addr learned my addr 172.21.15.118:0/2902240783 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 -- 172.21.15.118:0/2902240783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 msgr2=0x7fa858165bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858165bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 -- 172.21.15.118:0/2902240783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8580c0c50 msgr2=0x7fa858168fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8580c0c50 0x7fa858168fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 -- 172.21.15.118:0/2902240783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85000bc30 con 0x7fa85814bf60 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85ecfc700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa85814bf60 0x7fa85806a8b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c590 tx=0x7fa840007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa8567fc700 1 -- 172.21.15.118:0/2902240783 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840005a70 con 0x7fa85814bf60 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa8567fc700 1 -- 172.21.15.118:0/2902240783 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840005bd0 con 0x7fa85814bf60 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa858168b40 con 0x7fa85814bf60 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa8567fc700 1 -- 172.21.15.118:0/2902240783 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8400070d0 con 0x7fa85814bf60 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8581581b0 con 0x7fa85814bf60 2024-08-18T13:40:02.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.286+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8580713a0 con 0x7fa85814bf60 2024-08-18T13:40:02.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.290+0000 7fa8567fc700 1 -- 172.21.15.118:0/2902240783 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa8400074c0 con 0x7fa85814bf60 2024-08-18T13:40:02.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.290+0000 7fa8567fc700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa838060890 0x7fa838062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.290+0000 7fa8567fc700 1 -- 172.21.15.118:0/2902240783 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fa84007c200 con 0x7fa85814bf60 2024-08-18T13:40:02.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.290+0000 7fa85e4fb700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa838060890 0x7fa838062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.290+0000 7fa85e4fb700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa838060890 0x7fa838062d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa848007110 tx=0x7fa848007ea0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.298+0000 7fa8567fc700 1 -- 172.21.15.118:0/2902240783 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa84004c360 con 0x7fa85814bf60 2024-08-18T13:40:02.444 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.438+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa85805ecc0 con 0x7fa838060890 2024-08-18T13:40:02.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.454+0000 7fa8567fc700 1 -- 172.21.15.118:0/2902240783 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345211 (secure 0 0 0) 0x7fa85805ecc0 con 0x7fa838060890 2024-08-18T13:40:02.463 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:02.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa838060890 msgr2=0x7fa838062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa838060890 0x7fa838062d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa848007110 tx=0x7fa848007ea0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa85814bf60 msgr2=0x7fa85806a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa85814bf60 0x7fa85806a8b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c590 tx=0x7fa840007d40 comp rx=0 tx=0).stop 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 shutdown_connections 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa838060890 0x7fa838062d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa85814bf60 0x7fa85806a8b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8580c0c50 0x7fa858168fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 --2- 172.21.15.118:0/2902240783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa85805e6b0 0x7fa858165bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 >> 172.21.15.118:0/2902240783 conn(0x7fa8580c81e0 msgr2=0x7fa8580c3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 shutdown_connections 2024-08-18T13:40:02.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.462+0000 7fa85f4fd700 1 -- 172.21.15.118:0/2902240783 wait complete. 2024-08-18T13:40:02.468 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:40:02.477 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-08-18T13:40:01.205144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594916,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415368,"kb_used_data":3240,"kb_used_omap":13,"kb_used_meta":2412018,"kb_avail":752559352,"statfs":{"total":773094113280,"available":770620776448,"internally_reserved":0,"allocated":3317760,"data_stored":2006336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2469907136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1757,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005261"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433629+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:39:40.433629+0000","last_peered":"2024-08-18T13:39:40.433629+0000","last_clean":"2024-08-18T13:39:40.433629+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:39:40.433629+0000","last_undegraded":"2024-08-18T13:39:40.433629+0000","last_fullsized":"2024-08-18T13:39:40.433629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422872+0000","last_change":"2024-08-18T13:39:35.405966+0000","last_active":"2024-08-18T13:39:39.422872+0000","last_peered":"2024-08-18T13:39:39.422872+0000","last_clean":"2024-08-18T13:39:39.422872+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:39:39.422872+0000","last_undegraded":"2024-08-18T13:39:39.422872+0000","last_fullsized":"2024-08-18T13:39:39.422872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454715+0000","last_change":"2024-08-18T13:39:39.453674+0000","last_active":"2024-08-18T13:39:39.454715+0000","last_peered":"2024-08-18T13:39:39.454715+0000","last_clean":"2024-08-18T13:39:39.454715+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:39:39.454715+0000","last_undegraded":"2024-08-18T13:39:39.454715+0000","last_fullsized":"2024-08-18T13:39:39.454715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434410+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:39:40.434410+0000","last_peered":"2024-08-18T13:39:40.434410+0000","last_clean":"2024-08-18T13:39:40.434410+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:39:40.434410+0000","last_undegraded":"2024-08-18T13:39:40.434410+0000","last_fullsized":"2024-08-18T13:39:40.434410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432862+0000","last_change":"2024-08-18T13:39:35.408917+0000","last_active":"2024-08-18T13:39:39.432862+0000","last_peered":"2024-08-18T13:39:39.432862+0000","last_clean":"2024-08-18T13:39:39.432862+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:39:39.432862+0000","last_undegraded":"2024-08-18T13:39:39.432862+0000","last_fullsized":"2024-08-18T13:39:39.432862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447766+0000","last_change":"2024-08-18T13:39:39.445388+0000","last_active":"2024-08-18T13:39:39.447766+0000","last_peered":"2024-08-18T13:39:39.447766+0000","last_clean":"2024-08-18T13:39:39.447766+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:39:39.447766+0000","last_undegraded":"2024-08-18T13:39:39.447766+0000","last_fullsized":"2024-08-18T13:39:39.447766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430049+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:39:40.430049+0000","last_peered":"2024-08-18T13:39:40.430049+0000","last_clean":"2024-08-18T13:39:40.430049+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:39:40.430049+0000","last_undegraded":"2024-08-18T13:39:40.430049+0000","last_fullsized":"2024-08-18T13:39:40.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422649+0000","last_change":"2024-08-18T13:39:35.399188+0000","last_active":"2024-08-18T13:39:39.422649+0000","last_peered":"2024-08-18T13:39:39.422649+0000","last_clean":"2024-08-18T13:39:39.422649+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:39:39.422649+0000","last_undegraded":"2024-08-18T13:39:39.422649+0000","last_fullsized":"2024-08-18T13:39:39.422649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447285+0000","last_change":"2024-08-18T13:39:39.440922+0000","last_active":"2024-08-18T13:39:39.447285+0000","last_peered":"2024-08-18T13:39:39.447285+0000","last_clean":"2024-08-18T13:39:39.447285+0000","last_became_active":"2024-08-18T13:39:39.440426+0000","last_became_peered":"2024-08-18T13:39:39.440426+0000","last_unstale":"2024-08-18T13:39:39.447285+0000","last_undegraded":"2024-08-18T13:39:39.447285+0000","last_fullsized":"2024-08-18T13:39:39.447285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:23:00.376805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.428764+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:39:40.428764+0000","last_peered":"2024-08-18T13:39:40.428764+0000","last_clean":"2024-08-18T13:39:40.428764+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:39:40.428764+0000","last_undegraded":"2024-08-18T13:39:40.428764+0000","last_fullsized":"2024-08-18T13:39:40.428764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423184+0000","last_change":"2024-08-18T13:39:35.400038+0000","last_active":"2024-08-18T13:39:39.423184+0000","last_peered":"2024-08-18T13:39:39.423184+0000","last_clean":"2024-08-18T13:39:39.423184+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:39:39.423184+0000","last_undegraded":"2024-08-18T13:39:39.423184+0000","last_fullsized":"2024-08-18T13:39:39.423184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437377+0000","last_change":"2024-08-18T13:39:39.436221+0000","last_active":"2024-08-18T13:39:39.437377+0000","last_peered":"2024-08-18T13:39:39.437377+0000","last_clean":"2024-08-18T13:39:39.437377+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:39:39.437377+0000","last_undegraded":"2024-08-18T13:39:39.437377+0000","last_fullsized":"2024-08-18T13:39:39.437377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428676+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.428676+0000","last_peered":"2024-08-18T13:39:39.428676+0000","last_clean":"2024-08-18T13:39:39.428676+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:39:39.428676+0000","last_undegraded":"2024-08-18T13:39:39.428676+0000","last_fullsized":"2024-08-18T13:39:39.428676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430090+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:39:40.430090+0000","last_peered":"2024-08-18T13:39:40.430090+0000","last_clean":"2024-08-18T13:39:40.430090+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:39:40.430090+0000","last_undegraded":"2024-08-18T13:39:40.430090+0000","last_fullsized":"2024-08-18T13:39:40.430090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.443080+0000","last_change":"2024-08-18T13:39:39.439633+0000","last_active":"2024-08-18T13:39:39.443080+0000","last_peered":"2024-08-18T13:39:39.443080+0000","last_clean":"2024-08-18T13:39:39.443080+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:39:39.443080+0000","last_undegraded":"2024-08-18T13:39:39.443080+0000","last_fullsized":"2024-08-18T13:39:39.443080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423607+0000","last_change":"2024-08-18T13:39:35.407947+0000","last_active":"2024-08-18T13:39:39.423607+0000","last_peered":"2024-08-18T13:39:39.423607+0000","last_clean":"2024-08-18T13:39:39.423607+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:39:39.423607+0000","last_undegraded":"2024-08-18T13:39:39.423607+0000","last_fullsized":"2024-08-18T13:39:39.423607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429156+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:39:39.429156+0000","last_peered":"2024-08-18T13:39:39.429156+0000","last_clean":"2024-08-18T13:39:39.429156+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:39:39.429156+0000","last_undegraded":"2024-08-18T13:39:39.429156+0000","last_fullsized":"2024-08-18T13:39:39.429156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442892+0000","last_change":"2024-08-18T13:39:39.439913+0000","last_active":"2024-08-18T13:39:39.442892+0000","last_peered":"2024-08-18T13:39:39.442892+0000","last_clean":"2024-08-18T13:39:39.442892+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:39:39.442892+0000","last_undegraded":"2024-08-18T13:39:39.442892+0000","last_fullsized":"2024-08-18T13:39:39.442892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430332+0000","last_change":"2024-08-18T13:39:35.403254+0000","last_active":"2024-08-18T13:39:39.430332+0000","last_peered":"2024-08-18T13:39:39.430332+0000","last_clean":"2024-08-18T13:39:39.430332+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:39:39.430332+0000","last_undegraded":"2024-08-18T13:39:39.430332+0000","last_fullsized":"2024-08-18T13:39:39.430332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422689+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:39:39.422689+0000","last_peered":"2024-08-18T13:39:39.422689+0000","last_clean":"2024-08-18T13:39:39.422689+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:39:39.422689+0000","last_undegraded":"2024-08-18T13:39:39.422689+0000","last_fullsized":"2024-08-18T13:39:39.422689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457822+0000","last_change":"2024-08-18T13:39:39.457250+0000","last_active":"2024-08-18T13:39:39.457822+0000","last_peered":"2024-08-18T13:39:39.457822+0000","last_clean":"2024-08-18T13:39:39.457822+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:39:39.457822+0000","last_undegraded":"2024-08-18T13:39:39.457822+0000","last_fullsized":"2024-08-18T13:39:39.457822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425047+0000","last_change":"2024-08-18T13:39:35.401612+0000","last_active":"2024-08-18T13:39:39.425047+0000","last_peered":"2024-08-18T13:39:39.425047+0000","last_clean":"2024-08-18T13:39:39.425047+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:39:39.425047+0000","last_undegraded":"2024-08-18T13:39:39.425047+0000","last_fullsized":"2024-08-18T13:39:39.425047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450234+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:39:39.450234+0000","last_peered":"2024-08-18T13:39:39.450234+0000","last_clean":"2024-08-18T13:39:39.450234+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:39:39.450234+0000","last_undegraded":"2024-08-18T13:39:39.450234+0000","last_fullsized":"2024-08-18T13:39:39.450234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447688+0000","last_change":"2024-08-18T13:39:39.443503+0000","last_active":"2024-08-18T13:39:39.447688+0000","last_peered":"2024-08-18T13:39:39.447688+0000","last_clean":"2024-08-18T13:39:39.447688+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:39:39.447688+0000","last_undegraded":"2024-08-18T13:39:39.447688+0000","last_fullsized":"2024-08-18T13:39:39.447688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432976+0000","last_change":"2024-08-18T13:39:35.407907+0000","last_active":"2024-08-18T13:39:39.432976+0000","last_peered":"2024-08-18T13:39:39.432976+0000","last_clean":"2024-08-18T13:39:39.432976+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:39:39.432976+0000","last_undegraded":"2024-08-18T13:39:39.432976+0000","last_fullsized":"2024-08-18T13:39:39.432976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429181+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:39:40.429181+0000","last_peered":"2024-08-18T13:39:40.429181+0000","last_clean":"2024-08-18T13:39:40.429181+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:39:40.429181+0000","last_undegraded":"2024-08-18T13:39:40.429181+0000","last_fullsized":"2024-08-18T13:39:40.429181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454767+0000","last_change":"2024-08-18T13:39:39.453138+0000","last_active":"2024-08-18T13:39:39.454767+0000","last_peered":"2024-08-18T13:39:39.454767+0000","last_clean":"2024-08-18T13:39:39.454767+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:39:39.454767+0000","last_undegraded":"2024-08-18T13:39:39.454767+0000","last_fullsized":"2024-08-18T13:39:39.454767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424552+0000","last_change":"2024-08-18T13:39:35.404453+0000","last_active":"2024-08-18T13:39:39.424552+0000","last_peered":"2024-08-18T13:39:39.424552+0000","last_clean":"2024-08-18T13:39:39.424552+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:39:39.424552+0000","last_undegraded":"2024-08-18T13:39:39.424552+0000","last_fullsized":"2024-08-18T13:39:39.424552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:41.051281+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:39:41.051281+0000","last_peered":"2024-08-18T13:39:41.051281+0000","last_clean":"2024-08-18T13:39:41.051281+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:39:41.051281+0000","last_undegraded":"2024-08-18T13:39:41.051281+0000","last_fullsized":"2024-08-18T13:39:41.051281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450645+0000","last_change":"2024-08-18T13:39:39.449851+0000","last_active":"2024-08-18T13:39:39.450645+0000","last_peered":"2024-08-18T13:39:39.450645+0000","last_clean":"2024-08-18T13:39:39.450645+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:39:39.450645+0000","last_undegraded":"2024-08-18T13:39:39.450645+0000","last_fullsized":"2024-08-18T13:39:39.450645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424459+0000","last_change":"2024-08-18T13:39:35.400604+0000","last_active":"2024-08-18T13:39:39.424459+0000","last_peered":"2024-08-18T13:39:39.424459+0000","last_clean":"2024-08-18T13:39:39.424459+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:39:39.424459+0000","last_undegraded":"2024-08-18T13:39:39.424459+0000","last_fullsized":"2024-08-18T13:39:39.424459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450871+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:39:39.450871+0000","last_peered":"2024-08-18T13:39:39.450871+0000","last_clean":"2024-08-18T13:39:39.450871+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:39:39.450871+0000","last_undegraded":"2024-08-18T13:39:39.450871+0000","last_fullsized":"2024-08-18T13:39:39.450871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454096+0000","last_change":"2024-08-18T13:39:39.453599+0000","last_active":"2024-08-18T13:39:39.454096+0000","last_peered":"2024-08-18T13:39:39.454096+0000","last_clean":"2024-08-18T13:39:39.454096+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:39:39.454096+0000","last_undegraded":"2024-08-18T13:39:39.454096+0000","last_fullsized":"2024-08-18T13:39:39.454096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423160+0000","last_change":"2024-08-18T13:39:35.410167+0000","last_active":"2024-08-18T13:39:39.423160+0000","last_peered":"2024-08-18T13:39:39.423160+0000","last_clean":"2024-08-18T13:39:39.423160+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:39:39.423160+0000","last_undegraded":"2024-08-18T13:39:39.423160+0000","last_fullsized":"2024-08-18T13:39:39.423160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424551+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:39:39.424551+0000","last_peered":"2024-08-18T13:39:39.424551+0000","last_clean":"2024-08-18T13:39:39.424551+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:39:39.424551+0000","last_undegraded":"2024-08-18T13:39:39.424551+0000","last_fullsized":"2024-08-18T13:39:39.424551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444517+0000","last_change":"2024-08-18T13:39:39.439948+0000","last_active":"2024-08-18T13:39:39.444517+0000","last_peered":"2024-08-18T13:39:39.444517+0000","last_clean":"2024-08-18T13:39:39.444517+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:39:39.444517+0000","last_undegraded":"2024-08-18T13:39:39.444517+0000","last_fullsized":"2024-08-18T13:39:39.444517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422894+0000","last_change":"2024-08-18T13:39:35.407829+0000","last_active":"2024-08-18T13:39:39.422894+0000","last_peered":"2024-08-18T13:39:39.422894+0000","last_clean":"2024-08-18T13:39:39.422894+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:39:39.422894+0000","last_undegraded":"2024-08-18T13:39:39.422894+0000","last_fullsized":"2024-08-18T13:39:39.422894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417877+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:39:39.417877+0000","last_peered":"2024-08-18T13:39:39.417877+0000","last_clean":"2024-08-18T13:39:39.417877+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:39:39.417877+0000","last_undegraded":"2024-08-18T13:39:39.417877+0000","last_fullsized":"2024-08-18T13:39:39.417877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445479+0000","last_change":"2024-08-18T13:39:39.442372+0000","last_active":"2024-08-18T13:39:39.445479+0000","last_peered":"2024-08-18T13:39:39.445479+0000","last_clean":"2024-08-18T13:39:39.445479+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:39:39.445479+0000","last_undegraded":"2024-08-18T13:39:39.445479+0000","last_fullsized":"2024-08-18T13:39:39.445479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421862+0000","last_change":"2024-08-18T13:39:35.400384+0000","last_active":"2024-08-18T13:39:39.421862+0000","last_peered":"2024-08-18T13:39:39.421862+0000","last_clean":"2024-08-18T13:39:39.421862+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:39:39.421862+0000","last_undegraded":"2024-08-18T13:39:39.421862+0000","last_fullsized":"2024-08-18T13:39:39.421862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415832+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:39:39.415832+0000","last_peered":"2024-08-18T13:39:39.415832+0000","last_clean":"2024-08-18T13:39:39.415832+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:39:39.415832+0000","last_undegraded":"2024-08-18T13:39:39.415832+0000","last_fullsized":"2024-08-18T13:39:39.415832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446859+0000","last_change":"2024-08-18T13:39:39.446401+0000","last_active":"2024-08-18T13:39:39.446859+0000","last_peered":"2024-08-18T13:39:39.446859+0000","last_clean":"2024-08-18T13:39:39.446859+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:39:39.446859+0000","last_undegraded":"2024-08-18T13:39:39.446859+0000","last_fullsized":"2024-08-18T13:39:39.446859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424307+0000","last_change":"2024-08-18T13:39:35.410127+0000","last_active":"2024-08-18T13:39:39.424307+0000","last_peered":"2024-08-18T13:39:39.424307+0000","last_clean":"2024-08-18T13:39:39.424307+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:39:39.424307+0000","last_undegraded":"2024-08-18T13:39:39.424307+0000","last_fullsized":"2024-08-18T13:39:39.424307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450338+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:39:39.450338+0000","last_peered":"2024-08-18T13:39:39.450338+0000","last_clean":"2024-08-18T13:39:39.450338+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:39:39.450338+0000","last_undegraded":"2024-08-18T13:39:39.450338+0000","last_fullsized":"2024-08-18T13:39:39.450338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454861+0000","last_change":"2024-08-18T13:39:39.453901+0000","last_active":"2024-08-18T13:39:39.454861+0000","last_peered":"2024-08-18T13:39:39.454861+0000","last_clean":"2024-08-18T13:39:39.454861+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:39:39.454861+0000","last_undegraded":"2024-08-18T13:39:39.454861+0000","last_fullsized":"2024-08-18T13:39:39.454861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430338+0000","last_change":"2024-08-18T13:39:35.412156+0000","last_active":"2024-08-18T13:39:39.430338+0000","last_peered":"2024-08-18T13:39:39.430338+0000","last_clean":"2024-08-18T13:39:39.430338+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:39:39.430338+0000","last_undegraded":"2024-08-18T13:39:39.430338+0000","last_fullsized":"2024-08-18T13:39:39.430338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417339+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:39:39.417339+0000","last_peered":"2024-08-18T13:39:39.417339+0000","last_clean":"2024-08-18T13:39:39.417339+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:39:39.417339+0000","last_undegraded":"2024-08-18T13:39:39.417339+0000","last_fullsized":"2024-08-18T13:39:39.417339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450727+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:39:39.450727+0000","last_peered":"2024-08-18T13:39:39.450727+0000","last_clean":"2024-08-18T13:39:39.450727+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:39:39.450727+0000","last_undegraded":"2024-08-18T13:39:39.450727+0000","last_fullsized":"2024-08-18T13:39:39.450727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454332+0000","last_change":"2024-08-18T13:39:39.453360+0000","last_active":"2024-08-18T13:39:39.454332+0000","last_peered":"2024-08-18T13:39:39.454332+0000","last_clean":"2024-08-18T13:39:39.454332+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:39:39.454332+0000","last_undegraded":"2024-08-18T13:39:39.454332+0000","last_fullsized":"2024-08-18T13:39:39.454332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450556+0000","last_change":"2024-08-18T13:39:35.421765+0000","last_active":"2024-08-18T13:39:39.450556+0000","last_peered":"2024-08-18T13:39:39.450556+0000","last_clean":"2024-08-18T13:39:39.450556+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:39:39.450556+0000","last_undegraded":"2024-08-18T13:39:39.450556+0000","last_fullsized":"2024-08-18T13:39:39.450556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416404+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:39:39.416404+0000","last_peered":"2024-08-18T13:39:39.416404+0000","last_clean":"2024-08-18T13:39:39.416404+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:39:39.416404+0000","last_undegraded":"2024-08-18T13:39:39.416404+0000","last_fullsized":"2024-08-18T13:39:39.416404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450222+0000","last_change":"2024-08-18T13:39:39.449653+0000","last_active":"2024-08-18T13:39:39.450222+0000","last_peered":"2024-08-18T13:39:39.450222+0000","last_clean":"2024-08-18T13:39:39.450222+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:39:39.450222+0000","last_undegraded":"2024-08-18T13:39:39.450222+0000","last_fullsized":"2024-08-18T13:39:39.450222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422710+0000","last_change":"2024-08-18T13:39:35.407439+0000","last_active":"2024-08-18T13:39:39.422710+0000","last_peered":"2024-08-18T13:39:39.422710+0000","last_clean":"2024-08-18T13:39:39.422710+0000","last_became_active":"2024-08-18T13:39:35.407128+0000","last_became_peered":"2024-08-18T13:39:35.407128+0000","last_unstale":"2024-08-18T13:39:39.422710+0000","last_undegraded":"2024-08-18T13:39:39.422710+0000","last_fullsized":"2024-08-18T13:39:39.422710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431851+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:39:40.431851+0000","last_peered":"2024-08-18T13:39:40.431851+0000","last_clean":"2024-08-18T13:39:40.431851+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:39:40.431851+0000","last_undegraded":"2024-08-18T13:39:40.431851+0000","last_fullsized":"2024-08-18T13:39:40.431851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445844+0000","last_change":"2024-08-18T13:39:39.443848+0000","last_active":"2024-08-18T13:39:39.445844+0000","last_peered":"2024-08-18T13:39:39.445844+0000","last_clean":"2024-08-18T13:39:39.445844+0000","last_became_active":"2024-08-18T13:39:39.443456+0000","last_became_peered":"2024-08-18T13:39:39.443456+0000","last_unstale":"2024-08-18T13:39:39.445844+0000","last_undegraded":"2024-08-18T13:39:39.445844+0000","last_fullsized":"2024-08-18T13:39:39.445844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:27.994932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450757+0000","last_change":"2024-08-18T13:39:35.421612+0000","last_active":"2024-08-18T13:39:39.450757+0000","last_peered":"2024-08-18T13:39:39.450757+0000","last_clean":"2024-08-18T13:39:39.450757+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:39:39.450757+0000","last_undegraded":"2024-08-18T13:39:39.450757+0000","last_fullsized":"2024-08-18T13:39:39.450757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433442+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:39:40.433442+0000","last_peered":"2024-08-18T13:39:40.433442+0000","last_clean":"2024-08-18T13:39:40.433442+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:39:40.433442+0000","last_undegraded":"2024-08-18T13:39:40.433442+0000","last_fullsized":"2024-08-18T13:39:40.433442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454815+0000","last_change":"2024-08-18T13:39:39.453520+0000","last_active":"2024-08-18T13:39:39.454815+0000","last_peered":"2024-08-18T13:39:39.454815+0000","last_clean":"2024-08-18T13:39:39.454815+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:39:39.454815+0000","last_undegraded":"2024-08-18T13:39:39.454815+0000","last_fullsized":"2024-08-18T13:39:39.454815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450976+0000","last_change":"2024-08-18T13:39:35.421465+0000","last_active":"2024-08-18T13:39:39.450976+0000","last_peered":"2024-08-18T13:39:39.450976+0000","last_clean":"2024-08-18T13:39:39.450976+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:39:39.450976+0000","last_undegraded":"2024-08-18T13:39:39.450976+0000","last_fullsized":"2024-08-18T13:39:39.450976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429394+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:39:40.429394+0000","last_peered":"2024-08-18T13:39:40.429394+0000","last_clean":"2024-08-18T13:39:40.429394+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:39:40.429394+0000","last_undegraded":"2024-08-18T13:39:40.429394+0000","last_fullsized":"2024-08-18T13:39:40.429394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.448067+0000","last_change":"2024-08-18T13:39:39.446029+0000","last_active":"2024-08-18T13:39:39.448067+0000","last_peered":"2024-08-18T13:39:39.448067+0000","last_clean":"2024-08-18T13:39:39.448067+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:39:39.448067+0000","last_undegraded":"2024-08-18T13:39:39.448067+0000","last_fullsized":"2024-08-18T13:39:39.448067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450468+0000","last_change":"2024-08-18T13:39:35.417094+0000","last_active":"2024-08-18T13:39:39.450468+0000","last_peered":"2024-08-18T13:39:39.450468+0000","last_clean":"2024-08-18T13:39:39.450468+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:39:39.450468+0000","last_undegraded":"2024-08-18T13:39:39.450468+0000","last_fullsized":"2024-08-18T13:39:39.450468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.436721+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:39:40.436721+0000","last_peered":"2024-08-18T13:39:40.436721+0000","last_clean":"2024-08-18T13:39:40.436721+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:39:40.436721+0000","last_undegraded":"2024-08-18T13:39:40.436721+0000","last_fullsized":"2024-08-18T13:39:40.436721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437633+0000","last_change":"2024-08-18T13:39:39.436992+0000","last_active":"2024-08-18T13:39:39.437633+0000","last_peered":"2024-08-18T13:39:39.437633+0000","last_clean":"2024-08-18T13:39:39.437633+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:39:39.437633+0000","last_undegraded":"2024-08-18T13:39:39.437633+0000","last_fullsized":"2024-08-18T13:39:39.437633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427745+0000","last_change":"2024-08-18T13:39:35.411315+0000","last_active":"2024-08-18T13:39:39.427745+0000","last_peered":"2024-08-18T13:39:39.427745+0000","last_clean":"2024-08-18T13:39:39.427745+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:39:39.427745+0000","last_undegraded":"2024-08-18T13:39:39.427745+0000","last_fullsized":"2024-08-18T13:39:39.427745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428272+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:39:39.428272+0000","last_peered":"2024-08-18T13:39:39.428272+0000","last_clean":"2024-08-18T13:39:39.428272+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:39:39.428272+0000","last_undegraded":"2024-08-18T13:39:39.428272+0000","last_fullsized":"2024-08-18T13:39:39.428272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.458299+0000","last_change":"2024-08-18T13:39:39.456955+0000","last_active":"2024-08-18T13:39:39.458299+0000","last_peered":"2024-08-18T13:39:39.458299+0000","last_clean":"2024-08-18T13:39:39.458299+0000","last_became_active":"2024-08-18T13:39:39.456534+0000","last_became_peered":"2024-08-18T13:39:39.456534+0000","last_unstale":"2024-08-18T13:39:39.458299+0000","last_undegraded":"2024-08-18T13:39:39.458299+0000","last_fullsized":"2024-08-18T13:39:39.458299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:26:42.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450840+0000","last_change":"2024-08-18T13:39:35.417348+0000","last_active":"2024-08-18T13:39:39.450840+0000","last_peered":"2024-08-18T13:39:39.450840+0000","last_clean":"2024-08-18T13:39:39.450840+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:39:39.450840+0000","last_undegraded":"2024-08-18T13:39:39.450840+0000","last_fullsized":"2024-08-18T13:39:39.450840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417725+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:39:39.417725+0000","last_peered":"2024-08-18T13:39:39.417725+0000","last_clean":"2024-08-18T13:39:39.417725+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:39:39.417725+0000","last_undegraded":"2024-08-18T13:39:39.417725+0000","last_fullsized":"2024-08-18T13:39:39.417725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454724+0000","last_change":"2024-08-18T13:39:39.453682+0000","last_active":"2024-08-18T13:39:39.454724+0000","last_peered":"2024-08-18T13:39:39.454724+0000","last_clean":"2024-08-18T13:39:39.454724+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:39:39.454724+0000","last_undegraded":"2024-08-18T13:39:39.454724+0000","last_fullsized":"2024-08-18T13:39:39.454724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427706+0000","last_change":"2024-08-18T13:39:35.407127+0000","last_active":"2024-08-18T13:39:39.427706+0000","last_peered":"2024-08-18T13:39:39.427706+0000","last_clean":"2024-08-18T13:39:39.427706+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:39:39.427706+0000","last_undegraded":"2024-08-18T13:39:39.427706+0000","last_fullsized":"2024-08-18T13:39:39.427706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431523+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:39:40.431523+0000","last_peered":"2024-08-18T13:39:40.431523+0000","last_clean":"2024-08-18T13:39:40.431523+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:39:40.431523+0000","last_undegraded":"2024-08-18T13:39:40.431523+0000","last_fullsized":"2024-08-18T13:39:40.431523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457313+0000","last_change":"2024-08-18T13:39:39.456626+0000","last_active":"2024-08-18T13:39:39.457313+0000","last_peered":"2024-08-18T13:39:39.457313+0000","last_clean":"2024-08-18T13:39:39.457313+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:39:39.457313+0000","last_undegraded":"2024-08-18T13:39:39.457313+0000","last_fullsized":"2024-08-18T13:39:39.457313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450395+0000","last_change":"2024-08-18T13:39:35.417693+0000","last_active":"2024-08-18T13:39:39.450395+0000","last_peered":"2024-08-18T13:39:39.450395+0000","last_clean":"2024-08-18T13:39:39.450395+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:39:39.450395+0000","last_undegraded":"2024-08-18T13:39:39.450395+0000","last_fullsized":"2024-08-18T13:39:39.450395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:48.063001+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:39:48.063001+0000","last_peered":"2024-08-18T13:39:48.063001+0000","last_clean":"2024-08-18T13:39:48.063001+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:39:48.063001+0000","last_undegraded":"2024-08-18T13:39:48.063001+0000","last_fullsized":"2024-08-18T13:39:48.063001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3762,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.440639+0000","last_change":"2024-08-18T13:39:39.438064+0000","last_active":"2024-08-18T13:39:39.440639+0000","last_peered":"2024-08-18T13:39:39.440639+0000","last_clean":"2024-08-18T13:39:39.440639+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:39:39.440639+0000","last_undegraded":"2024-08-18T13:39:39.440639+0000","last_fullsized":"2024-08-18T13:39:39.440639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421839+0000","last_change":"2024-08-18T13:39:35.406398+0000","last_active":"2024-08-18T13:39:39.421839+0000","last_peered":"2024-08-18T13:39:39.421839+0000","last_clean":"2024-08-18T13:39:39.421839+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:39:39.421839+0000","last_undegraded":"2024-08-18T13:39:39.421839+0000","last_fullsized":"2024-08-18T13:39:39.421839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433257+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:39:40.433257+0000","last_peered":"2024-08-18T13:39:40.433257+0000","last_clean":"2024-08-18T13:39:40.433257+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:39:40.433257+0000","last_undegraded":"2024-08-18T13:39:40.433257+0000","last_fullsized":"2024-08-18T13:39:40.433257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444385+0000","last_change":"2024-08-18T13:39:39.439962+0000","last_active":"2024-08-18T13:39:39.444385+0000","last_peered":"2024-08-18T13:39:39.444385+0000","last_clean":"2024-08-18T13:39:39.444385+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:39:39.444385+0000","last_undegraded":"2024-08-18T13:39:39.444385+0000","last_fullsized":"2024-08-18T13:39:39.444385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432863+0000","last_change":"2024-08-18T13:39:35.411067+0000","last_active":"2024-08-18T13:39:39.432863+0000","last_peered":"2024-08-18T13:39:39.432863+0000","last_clean":"2024-08-18T13:39:39.432863+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:39:39.432863+0000","last_undegraded":"2024-08-18T13:39:39.432863+0000","last_fullsized":"2024-08-18T13:39:39.432863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422811+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:39:39.422811+0000","last_peered":"2024-08-18T13:39:39.422811+0000","last_clean":"2024-08-18T13:39:39.422811+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:39:39.422811+0000","last_undegraded":"2024-08-18T13:39:39.422811+0000","last_fullsized":"2024-08-18T13:39:39.422811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447069+0000","last_change":"2024-08-18T13:39:39.443938+0000","last_active":"2024-08-18T13:39:39.447069+0000","last_peered":"2024-08-18T13:39:39.447069+0000","last_clean":"2024-08-18T13:39:39.447069+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:39:39.447069+0000","last_undegraded":"2024-08-18T13:39:39.447069+0000","last_fullsized":"2024-08-18T13:39:39.447069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422769+0000","last_change":"2024-08-18T13:39:35.409517+0000","last_active":"2024-08-18T13:39:39.422769+0000","last_peered":"2024-08-18T13:39:39.422769+0000","last_clean":"2024-08-18T13:39:39.422769+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:39:39.422769+0000","last_undegraded":"2024-08-18T13:39:39.422769+0000","last_fullsized":"2024-08-18T13:39:39.422769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429841+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:39:39.429841+0000","last_peered":"2024-08-18T13:39:39.429841+0000","last_clean":"2024-08-18T13:39:39.429841+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:39:39.429841+0000","last_undegraded":"2024-08-18T13:39:39.429841+0000","last_fullsized":"2024-08-18T13:39:39.429841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450637+0000","last_change":"2024-08-18T13:39:39.449671+0000","last_active":"2024-08-18T13:39:39.450637+0000","last_peered":"2024-08-18T13:39:39.450637+0000","last_clean":"2024-08-18T13:39:39.450637+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:39:39.450637+0000","last_undegraded":"2024-08-18T13:39:39.450637+0000","last_fullsized":"2024-08-18T13:39:39.450637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454824+0000","last_change":"2024-08-18T13:39:39.453907+0000","last_active":"2024-08-18T13:39:39.454824+0000","last_peered":"2024-08-18T13:39:39.454824+0000","last_clean":"2024-08-18T13:39:39.454824+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:39:39.454824+0000","last_undegraded":"2024-08-18T13:39:39.454824+0000","last_fullsized":"2024-08-18T13:39:39.454824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422264+0000","last_change":"2024-08-18T13:39:35.401997+0000","last_active":"2024-08-18T13:39:39.422264+0000","last_peered":"2024-08-18T13:39:39.422264+0000","last_clean":"2024-08-18T13:39:39.422264+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:39:39.422264+0000","last_undegraded":"2024-08-18T13:39:39.422264+0000","last_fullsized":"2024-08-18T13:39:39.422264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418448+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:39:39.418448+0000","last_peered":"2024-08-18T13:39:39.418448+0000","last_clean":"2024-08-18T13:39:39.418448+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:39:39.418448+0000","last_undegraded":"2024-08-18T13:39:39.418448+0000","last_fullsized":"2024-08-18T13:39:39.418448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442977+0000","last_change":"2024-08-18T13:39:39.440811+0000","last_active":"2024-08-18T13:39:39.442977+0000","last_peered":"2024-08-18T13:39:39.442977+0000","last_clean":"2024-08-18T13:39:39.442977+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:39:39.442977+0000","last_undegraded":"2024-08-18T13:39:39.442977+0000","last_fullsized":"2024-08-18T13:39:39.442977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430210+0000","last_change":"2024-08-18T13:39:35.412127+0000","last_active":"2024-08-18T13:39:39.430210+0000","last_peered":"2024-08-18T13:39:39.430210+0000","last_clean":"2024-08-18T13:39:39.430210+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:39:39.430210+0000","last_undegraded":"2024-08-18T13:39:39.430210+0000","last_fullsized":"2024-08-18T13:39:39.430210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423847+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:39:39.423847+0000","last_peered":"2024-08-18T13:39:39.423847+0000","last_clean":"2024-08-18T13:39:39.423847+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:39:39.423847+0000","last_undegraded":"2024-08-18T13:39:39.423847+0000","last_fullsized":"2024-08-18T13:39:39.423847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429678+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:39:40.429678+0000","last_peered":"2024-08-18T13:39:40.429678+0000","last_clean":"2024-08-18T13:39:40.429678+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:39:40.429678+0000","last_undegraded":"2024-08-18T13:39:40.429678+0000","last_fullsized":"2024-08-18T13:39:40.429678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429574+0000","last_change":"2024-08-18T13:39:35.407995+0000","last_active":"2024-08-18T13:39:39.429574+0000","last_peered":"2024-08-18T13:39:39.429574+0000","last_clean":"2024-08-18T13:39:39.429574+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:39:39.429574+0000","last_undegraded":"2024-08-18T13:39:39.429574+0000","last_fullsized":"2024-08-18T13:39:39.429574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.438294+0000","last_change":"2024-08-18T13:39:39.437542+0000","last_active":"2024-08-18T13:39:39.438294+0000","last_peered":"2024-08-18T13:39:39.438294+0000","last_clean":"2024-08-18T13:39:39.438294+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:39:39.438294+0000","last_undegraded":"2024-08-18T13:39:39.438294+0000","last_fullsized":"2024-08-18T13:39:39.438294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434310+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:39:40.434310+0000","last_peered":"2024-08-18T13:39:40.434310+0000","last_clean":"2024-08-18T13:39:40.434310+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:39:40.434310+0000","last_undegraded":"2024-08-18T13:39:40.434310+0000","last_fullsized":"2024-08-18T13:39:40.434310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422760+0000","last_change":"2024-08-18T13:39:35.403759+0000","last_active":"2024-08-18T13:39:39.422760+0000","last_peered":"2024-08-18T13:39:39.422760+0000","last_clean":"2024-08-18T13:39:39.422760+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:39:39.422760+0000","last_undegraded":"2024-08-18T13:39:39.422760+0000","last_fullsized":"2024-08-18T13:39:39.422760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446432+0000","last_change":"2024-08-18T13:39:39.441201+0000","last_active":"2024-08-18T13:39:39.446432+0000","last_peered":"2024-08-18T13:39:39.446432+0000","last_clean":"2024-08-18T13:39:39.446432+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:39:39.446432+0000","last_undegraded":"2024-08-18T13:39:39.446432+0000","last_fullsized":"2024-08-18T13:39:39.446432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427616+0000","last_change":"2024-08-18T13:39:35.413292+0000","last_active":"2024-08-18T13:39:39.427616+0000","last_peered":"2024-08-18T13:39:39.427616+0000","last_clean":"2024-08-18T13:39:39.427616+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:39:39.427616+0000","last_undegraded":"2024-08-18T13:39:39.427616+0000","last_fullsized":"2024-08-18T13:39:39.427616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415812+0000","last_change":"2024-08-18T13:39:35.401196+0000","last_active":"2024-08-18T13:39:39.415812+0000","last_peered":"2024-08-18T13:39:39.415812+0000","last_clean":"2024-08-18T13:39:39.415812+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:39:39.415812+0000","last_undegraded":"2024-08-18T13:39:39.415812+0000","last_fullsized":"2024-08-18T13:39:39.415812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417577+0000","last_change":"2024-08-18T13:39:35.406603+0000","last_active":"2024-08-18T13:39:39.417577+0000","last_peered":"2024-08-18T13:39:39.417577+0000","last_clean":"2024-08-18T13:39:39.417577+0000","last_became_active":"2024-08-18T13:39:35.406331+0000","last_became_peered":"2024-08-18T13:39:35.406331+0000","last_unstale":"2024-08-18T13:39:39.417577+0000","last_undegraded":"2024-08-18T13:39:39.417577+0000","last_fullsized":"2024-08-18T13:39:39.417577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:21:12.232172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429494+0000","last_change":"2024-08-18T13:39:35.404051+0000","last_active":"2024-08-18T13:39:39.429494+0000","last_peered":"2024-08-18T13:39:39.429494+0000","last_clean":"2024-08-18T13:39:39.429494+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:39:39.429494+0000","last_undegraded":"2024-08-18T13:39:39.429494+0000","last_fullsized":"2024-08-18T13:39:39.429494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422959+0000","last_change":"2024-08-18T13:39:35.409288+0000","last_active":"2024-08-18T13:39:39.422959+0000","last_peered":"2024-08-18T13:39:39.422959+0000","last_clean":"2024-08-18T13:39:39.422959+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:39:39.422959+0000","last_undegraded":"2024-08-18T13:39:39.422959+0000","last_fullsized":"2024-08-18T13:39:39.422959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424121+0000","last_change":"2024-08-18T13:39:35.403051+0000","last_active":"2024-08-18T13:39:39.424121+0000","last_peered":"2024-08-18T13:39:39.424121+0000","last_clean":"2024-08-18T13:39:39.424121+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:39:39.424121+0000","last_undegraded":"2024-08-18T13:39:39.424121+0000","last_fullsized":"2024-08-18T13:39:39.424121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430049+0000","last_change":"2024-08-18T13:39:35.412281+0000","last_active":"2024-08-18T13:39:39.430049+0000","last_peered":"2024-08-18T13:39:39.430049+0000","last_clean":"2024-08-18T13:39:39.430049+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:39:39.430049+0000","last_undegraded":"2024-08-18T13:39:39.430049+0000","last_fullsized":"2024-08-18T13:39:39.430049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424466+0000","last_change":"2024-08-18T13:39:35.409220+0000","last_active":"2024-08-18T13:39:39.424466+0000","last_peered":"2024-08-18T13:39:39.424466+0000","last_clean":"2024-08-18T13:39:39.424466+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:39:39.424466+0000","last_undegraded":"2024-08-18T13:39:39.424466+0000","last_fullsized":"2024-08-18T13:39:39.424466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418253+0000","last_change":"2024-08-18T13:39:35.401759+0000","last_active":"2024-08-18T13:39:39.418253+0000","last_peered":"2024-08-18T13:39:39.418253+0000","last_clean":"2024-08-18T13:39:39.418253+0000","last_became_active":"2024-08-18T13:39:35.401476+0000","last_became_peered":"2024-08-18T13:39:35.401476+0000","last_unstale":"2024-08-18T13:39:39.418253+0000","last_undegraded":"2024-08-18T13:39:39.418253+0000","last_fullsized":"2024-08-18T13:39:39.418253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:04:43.997597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423669+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.423669+0000","last_peered":"2024-08-18T13:39:39.423669+0000","last_clean":"2024-08-18T13:39:39.423669+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:39:39.423669+0000","last_undegraded":"2024-08-18T13:39:39.423669+0000","last_fullsized":"2024-08-18T13:39:39.423669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417410+0000","last_change":"2024-08-18T13:39:35.407218+0000","last_active":"2024-08-18T13:39:39.417410+0000","last_peered":"2024-08-18T13:39:39.417410+0000","last_clean":"2024-08-18T13:39:39.417410+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:39:39.417410+0000","last_undegraded":"2024-08-18T13:39:39.417410+0000","last_fullsized":"2024-08-18T13:39:39.417410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429387+0000","last_change":"2024-08-18T13:39:35.407339+0000","last_active":"2024-08-18T13:39:39.429387+0000","last_peered":"2024-08-18T13:39:39.429387+0000","last_clean":"2024-08-18T13:39:39.429387+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:39:39.429387+0000","last_undegraded":"2024-08-18T13:39:39.429387+0000","last_fullsized":"2024-08-18T13:39:39.429387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429062+0000","last_change":"2024-08-18T13:39:35.408647+0000","last_active":"2024-08-18T13:39:39.429062+0000","last_peered":"2024-08-18T13:39:39.429062+0000","last_clean":"2024-08-18T13:39:39.429062+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:39:39.429062+0000","last_undegraded":"2024-08-18T13:39:39.429062+0000","last_fullsized":"2024-08-18T13:39:39.429062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450837+0000","last_change":"2024-08-18T13:39:35.418000+0000","last_active":"2024-08-18T13:39:39.450837+0000","last_peered":"2024-08-18T13:39:39.450837+0000","last_clean":"2024-08-18T13:39:39.450837+0000","last_became_active":"2024-08-18T13:39:35.417661+0000","last_became_peered":"2024-08-18T13:39:35.417661+0000","last_unstale":"2024-08-18T13:39:39.450837+0000","last_undegraded":"2024-08-18T13:39:39.450837+0000","last_fullsized":"2024-08-18T13:39:39.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:53:44.739585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416124+0000","last_change":"2024-08-18T13:39:35.405425+0000","last_active":"2024-08-18T13:39:39.416124+0000","last_peered":"2024-08-18T13:39:39.416124+0000","last_clean":"2024-08-18T13:39:39.416124+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:39:39.416124+0000","last_undegraded":"2024-08-18T13:39:39.416124+0000","last_fullsized":"2024-08-18T13:39:39.416124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423886+0000","last_change":"2024-08-18T13:39:35.400889+0000","last_active":"2024-08-18T13:39:39.423886+0000","last_peered":"2024-08-18T13:39:39.423886+0000","last_clean":"2024-08-18T13:39:39.423886+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:39:39.423886+0000","last_undegraded":"2024-08-18T13:39:39.423886+0000","last_fullsized":"2024-08-18T13:39:39.423886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418057+0000","last_change":"2024-08-18T13:39:35.407634+0000","last_active":"2024-08-18T13:39:39.418057+0000","last_peered":"2024-08-18T13:39:39.418057+0000","last_clean":"2024-08-18T13:39:39.418057+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:39:39.418057+0000","last_undegraded":"2024-08-18T13:39:39.418057+0000","last_fullsized":"2024-08-18T13:39:39.418057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422473+0000","last_change":"2024-08-18T13:39:35.397509+0000","last_active":"2024-08-18T13:39:39.422473+0000","last_peered":"2024-08-18T13:39:39.422473+0000","last_clean":"2024-08-18T13:39:39.422473+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:39:39.422473+0000","last_undegraded":"2024-08-18T13:39:39.422473+0000","last_fullsized":"2024-08-18T13:39:39.422473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425330+0000","last_change":"2024-08-18T13:39:35.404533+0000","last_active":"2024-08-18T13:39:39.425330+0000","last_peered":"2024-08-18T13:39:39.425330+0000","last_clean":"2024-08-18T13:39:39.425330+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:39:39.425330+0000","last_undegraded":"2024-08-18T13:39:39.425330+0000","last_fullsized":"2024-08-18T13:39:39.425330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425567+0000","last_change":"2024-08-18T13:39:35.404839+0000","last_active":"2024-08-18T13:39:39.425567+0000","last_peered":"2024-08-18T13:39:39.425567+0000","last_clean":"2024-08-18T13:39:39.425567+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:39:39.425567+0000","last_undegraded":"2024-08-18T13:39:39.425567+0000","last_fullsized":"2024-08-18T13:39:39.425567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421633+0000","last_change":"2024-08-18T13:39:35.405867+0000","last_active":"2024-08-18T13:39:39.421633+0000","last_peered":"2024-08-18T13:39:39.421633+0000","last_clean":"2024-08-18T13:39:39.421633+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:39:39.421633+0000","last_undegraded":"2024-08-18T13:39:39.421633+0000","last_fullsized":"2024-08-18T13:39:39.421633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429697+0000","last_change":"2024-08-18T13:39:35.411173+0000","last_active":"2024-08-18T13:39:39.429697+0000","last_peered":"2024-08-18T13:39:39.429697+0000","last_clean":"2024-08-18T13:39:39.429697+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:39:39.429697+0000","last_undegraded":"2024-08-18T13:39:39.429697+0000","last_fullsized":"2024-08-18T13:39:39.429697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429962+0000","last_change":"2024-08-18T13:39:35.418013+0000","last_active":"2024-08-18T13:39:39.429962+0000","last_peered":"2024-08-18T13:39:39.429962+0000","last_clean":"2024-08-18T13:39:39.429962+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:39:39.429962+0000","last_undegraded":"2024-08-18T13:39:39.429962+0000","last_fullsized":"2024-08-18T13:39:39.429962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450803+0000","last_change":"2024-08-18T13:39:35.421633+0000","last_active":"2024-08-18T13:39:39.450803+0000","last_peered":"2024-08-18T13:39:39.450803+0000","last_clean":"2024-08-18T13:39:39.450803+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:39:39.450803+0000","last_undegraded":"2024-08-18T13:39:39.450803+0000","last_fullsized":"2024-08-18T13:39:39.450803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425234+0000","last_change":"2024-08-18T13:39:35.411443+0000","last_active":"2024-08-18T13:39:39.425234+0000","last_peered":"2024-08-18T13:39:39.425234+0000","last_clean":"2024-08-18T13:39:39.425234+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:39:39.425234+0000","last_undegraded":"2024-08-18T13:39:39.425234+0000","last_fullsized":"2024-08-18T13:39:39.425234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450366+0000","last_change":"2024-08-18T13:39:35.415548+0000","last_active":"2024-08-18T13:39:39.450366+0000","last_peered":"2024-08-18T13:39:39.450366+0000","last_clean":"2024-08-18T13:39:39.450366+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:39:39.450366+0000","last_undegraded":"2024-08-18T13:39:39.450366+0000","last_fullsized":"2024-08-18T13:39:39.450366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417246+0000","last_change":"2024-08-18T13:39:35.410795+0000","last_active":"2024-08-18T13:39:39.417246+0000","last_peered":"2024-08-18T13:39:39.417246+0000","last_clean":"2024-08-18T13:39:39.417246+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:39:39.417246+0000","last_undegraded":"2024-08-18T13:39:39.417246+0000","last_fullsized":"2024-08-18T13:39:39.417246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450710+0000","last_change":"2024-08-18T13:39:35.417550+0000","last_active":"2024-08-18T13:39:39.450710+0000","last_peered":"2024-08-18T13:39:39.450710+0000","last_clean":"2024-08-18T13:39:39.450710+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:39:39.450710+0000","last_undegraded":"2024-08-18T13:39:39.450710+0000","last_fullsized":"2024-08-18T13:39:39.450710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422555+0000","last_change":"2024-08-18T13:39:35.410002+0000","last_active":"2024-08-18T13:39:39.422555+0000","last_peered":"2024-08-18T13:39:39.422555+0000","last_clean":"2024-08-18T13:39:39.422555+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:39:39.422555+0000","last_undegraded":"2024-08-18T13:39:39.422555+0000","last_fullsized":"2024-08-18T13:39:39.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427751+0000","last_change":"2024-08-18T13:39:35.406627+0000","last_active":"2024-08-18T13:39:39.427751+0000","last_peered":"2024-08-18T13:39:39.427751+0000","last_clean":"2024-08-18T13:39:39.427751+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:39:39.427751+0000","last_undegraded":"2024-08-18T13:39:39.427751+0000","last_fullsized":"2024-08-18T13:39:39.427751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423078+0000","last_change":"2024-08-18T13:39:35.405750+0000","last_active":"2024-08-18T13:39:39.423078+0000","last_peered":"2024-08-18T13:39:39.423078+0000","last_clean":"2024-08-18T13:39:39.423078+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:39:39.423078+0000","last_undegraded":"2024-08-18T13:39:39.423078+0000","last_fullsized":"2024-08-18T13:39:39.423078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450591+0000","last_change":"2024-08-18T13:39:35.417195+0000","last_active":"2024-08-18T13:39:39.450591+0000","last_peered":"2024-08-18T13:39:39.450591+0000","last_clean":"2024-08-18T13:39:39.450591+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:39:39.450591+0000","last_undegraded":"2024-08-18T13:39:39.450591+0000","last_fullsized":"2024-08-18T13:39:39.450591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4548,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":856064,"data_stored":693001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]}]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]}]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":270336,"data_stored":102189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]}]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":258048,"data_stored":102063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":102223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":864256,"data_stored":693057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":262144,"data_stored":102153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:40:02.482 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:40:02.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.578+0000 7fe004b06700 1 Processor -- start 2024-08-18T13:40:02.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.578+0000 7fe004b06700 1 -- start start 2024-08-18T13:40:02.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe000069810 0x7fe0000673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe000152f40 0x7fe0001533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000c20c0 con 0x7fe000069be0 2024-08-18T13:40:02.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0000c2960 con 0x7fe000069810 2024-08-18T13:40:02.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe000067910 con 0x7fe000152f40 2024-08-18T13:40:02.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe000069810 0x7fe0000673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe000069810 0x7fe0000673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45442/0 (socket says 172.21.15.118:45442) 2024-08-18T13:40:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 -- 172.21.15.118:0/1672628982 learned_addr learned my addr 172.21.15.118:0/1672628982 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdffffff700 1 --2- 172.21.15.118:0/1672628982 >> v2:172.21.15.178:3300/0 conn(0x7fe000152f40 0x7fe0001533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 -- 172.21.15.118:0/1672628982 >> v2:172.21.15.178:3300/0 conn(0x7fe000152f40 msgr2=0x7fe0001533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 --2- 172.21.15.118:0/1672628982 >> v2:172.21.15.178:3300/0 conn(0x7fe000152f40 0x7fe0001533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 -- 172.21.15.118:0/1672628982 >> v1:172.21.15.178:6789/0 conn(0x7fe000069be0 legacy=0x7fe0000c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 -- 172.21.15.118:0/1672628982 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000c2df0 con 0x7fe000069810 2024-08-18T13:40:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdffffff700 1 --2- 172.21.15.118:0/1672628982 >> v2:172.21.15.178:3300/0 conn(0x7fe000152f40 0x7fe0001533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:02.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdfff7fe700 1 --2- 172.21.15.118:0/1672628982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000069810 0x7fe0000673d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe00006bcb0 tx=0x7fdff0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77aa9ca6657c3e9f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdffe7fc700 1 -- 172.21.15.118:0/1672628982 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000b860 con 0x7fe000069810 2024-08-18T13:40:02.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdffe7fc700 1 -- 172.21.15.118:0/1672628982 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff000b9c0 con 0x7fe000069810 2024-08-18T13:40:02.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fdffe7fc700 1 -- 172.21.15.118:0/1672628982 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000bd60 con 0x7fe000069810 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- 172.21.15.118:0/1672628982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000069810 msgr2=0x7fe0000673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 --2- 172.21.15.118:0/1672628982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000069810 0x7fe0000673d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe00006bcb0 tx=0x7fdff0009250 comp rx=0 tx=0).stop 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- 172.21.15.118:0/1672628982 shutdown_connections 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 --2- 172.21.15.118:0/1672628982 >> v2:172.21.15.178:3300/0 conn(0x7fe000152f40 0x7fe0001533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 --2- 172.21.15.118:0/1672628982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000069810 0x7fe0000673d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- 172.21.15.118:0/1672628982 >> 172.21.15.118:0/1672628982 conn(0x7fe00006f4d0 msgr2=0x7fe0000c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- 172.21.15.118:0/1672628982 shutdown_connections 2024-08-18T13:40:02.589 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.582+0000 7fe004b06700 1 -- 172.21.15.118:0/1672628982 wait complete. 2024-08-18T13:40:02.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 Processor -- start 2024-08-18T13:40:02.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 -- start start 2024-08-18T13:40:02.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe000069810 0x7fe0001637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe000069be0 0x7fe00014fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000152f40 0x7fe0000c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00014f350 con 0x7fe000152f40 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c77a0 con 0x7fe000069be0 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c7a80 con 0x7fe000069810 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000152f40 0x7fe0000c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000152f40 0x7fe0000c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45444/0 (socket says 172.21.15.118:45444) 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 -- 172.21.15.118:0/3214627536 learned_addr learned my addr 172.21.15.118:0/3214627536 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffeffd700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe000069be0 0x7fe00014fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdfff7fe700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe000069810 0x7fe0001637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 -- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe000069810 msgr2=0x7fe0001637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe000069810 0x7fe0001637c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 -- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe000069be0 msgr2=0x7fe00014fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe000069be0 0x7fe00014fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 -- 172.21.15.118:0/3214627536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00094f0 con 0x7fe000152f40 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdffffff700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000152f40 0x7fe0000c7f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdff800d100 tx=0x7fdff80099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdff77fe700 1 -- 172.21.15.118:0/3214627536 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800dda0 con 0x7fe000152f40 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdff77fe700 1 -- 172.21.15.118:0/3214627536 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff800f040 con 0x7fe000152f40 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe000163d00 con 0x7fe000152f40 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fdff77fe700 1 -- 172.21.15.118:0/3214627536 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8015690 con 0x7fe000152f40 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0000bf970 con 0x7fe000152f40 2024-08-18T13:40:02.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.586+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0000c7c10 con 0x7fe000152f40 2024-08-18T13:40:02.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.590+0000 7fdff77fe700 1 -- 172.21.15.118:0/3214627536 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdff801f440 con 0x7fe000152f40 2024-08-18T13:40:02.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.590+0000 7fdff77fe700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fdfe00608e0 0x7fdfe0062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.590+0000 7fdfff7fe700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fdfe00608e0 0x7fdfe0062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.590+0000 7fdff77fe700 1 -- 172.21.15.118:0/3214627536 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fdff807d3b0 con 0x7fe000152f40 2024-08-18T13:40:02.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.590+0000 7fdfff7fe700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fdfe00608e0 0x7fdfe0062d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe00006bcb0 tx=0x7fdff0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.598+0000 7fdff77fe700 1 -- 172.21.15.118:0/3214627536 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdff804d470 con 0x7fe000152f40 2024-08-18T13:40:02.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.746+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0000713a0 con 0x7fdfe00608e0 2024-08-18T13:40:02.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.762+0000 7fdff77fe700 1 -- 172.21.15.118:0/3214627536 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345211 (secure 0 0 0) 0x7fe0000713a0 con 0x7fdfe00608e0 2024-08-18T13:40:02.772 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fdfe00608e0 msgr2=0x7fdfe0062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fdfe00608e0 0x7fdfe0062d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe00006bcb0 tx=0x7fdff0017000 comp rx=0 tx=0).stop 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000152f40 msgr2=0x7fe0000c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000152f40 0x7fe0000c7f10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdff800d100 tx=0x7fdff80099b0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 shutdown_connections 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fdfe00608e0 0x7fdfe0062d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe000152f40 0x7fe0000c7f10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe000069be0 0x7fe00014fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 --2- 172.21.15.118:0/3214627536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe000069810 0x7fe0001637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 >> 172.21.15.118:0/3214627536 conn(0x7fe00006f4d0 msgr2=0x7fe0000c3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 shutdown_connections 2024-08-18T13:40:02.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.766+0000 7fe004b06700 1 -- 172.21.15.118:0/3214627536 wait complete. 2024-08-18T13:40:02.776 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:40:02.786 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-08-18T13:40:01.205144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594916,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415368,"kb_used_data":3240,"kb_used_omap":13,"kb_used_meta":2412018,"kb_avail":752559352,"statfs":{"total":773094113280,"available":770620776448,"internally_reserved":0,"allocated":3317760,"data_stored":2006336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2469907136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1757,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005261"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433629+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:39:40.433629+0000","last_peered":"2024-08-18T13:39:40.433629+0000","last_clean":"2024-08-18T13:39:40.433629+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:39:40.433629+0000","last_undegraded":"2024-08-18T13:39:40.433629+0000","last_fullsized":"2024-08-18T13:39:40.433629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422872+0000","last_change":"2024-08-18T13:39:35.405966+0000","last_active":"2024-08-18T13:39:39.422872+0000","last_peered":"2024-08-18T13:39:39.422872+0000","last_clean":"2024-08-18T13:39:39.422872+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:39:39.422872+0000","last_undegraded":"2024-08-18T13:39:39.422872+0000","last_fullsized":"2024-08-18T13:39:39.422872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454715+0000","last_change":"2024-08-18T13:39:39.453674+0000","last_active":"2024-08-18T13:39:39.454715+0000","last_peered":"2024-08-18T13:39:39.454715+0000","last_clean":"2024-08-18T13:39:39.454715+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:39:39.454715+0000","last_undegraded":"2024-08-18T13:39:39.454715+0000","last_fullsized":"2024-08-18T13:39:39.454715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434410+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:39:40.434410+0000","last_peered":"2024-08-18T13:39:40.434410+0000","last_clean":"2024-08-18T13:39:40.434410+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:39:40.434410+0000","last_undegraded":"2024-08-18T13:39:40.434410+0000","last_fullsized":"2024-08-18T13:39:40.434410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432862+0000","last_change":"2024-08-18T13:39:35.408917+0000","last_active":"2024-08-18T13:39:39.432862+0000","last_peered":"2024-08-18T13:39:39.432862+0000","last_clean":"2024-08-18T13:39:39.432862+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:39:39.432862+0000","last_undegraded":"2024-08-18T13:39:39.432862+0000","last_fullsized":"2024-08-18T13:39:39.432862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447766+0000","last_change":"2024-08-18T13:39:39.445388+0000","last_active":"2024-08-18T13:39:39.447766+0000","last_peered":"2024-08-18T13:39:39.447766+0000","last_clean":"2024-08-18T13:39:39.447766+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:39:39.447766+0000","last_undegraded":"2024-08-18T13:39:39.447766+0000","last_fullsized":"2024-08-18T13:39:39.447766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430049+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:39:40.430049+0000","last_peered":"2024-08-18T13:39:40.430049+0000","last_clean":"2024-08-18T13:39:40.430049+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:39:40.430049+0000","last_undegraded":"2024-08-18T13:39:40.430049+0000","last_fullsized":"2024-08-18T13:39:40.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422649+0000","last_change":"2024-08-18T13:39:35.399188+0000","last_active":"2024-08-18T13:39:39.422649+0000","last_peered":"2024-08-18T13:39:39.422649+0000","last_clean":"2024-08-18T13:39:39.422649+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:39:39.422649+0000","last_undegraded":"2024-08-18T13:39:39.422649+0000","last_fullsized":"2024-08-18T13:39:39.422649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447285+0000","last_change":"2024-08-18T13:39:39.440922+0000","last_active":"2024-08-18T13:39:39.447285+0000","last_peered":"2024-08-18T13:39:39.447285+0000","last_clean":"2024-08-18T13:39:39.447285+0000","last_became_active":"2024-08-18T13:39:39.440426+0000","last_became_peered":"2024-08-18T13:39:39.440426+0000","last_unstale":"2024-08-18T13:39:39.447285+0000","last_undegraded":"2024-08-18T13:39:39.447285+0000","last_fullsized":"2024-08-18T13:39:39.447285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:23:00.376805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.428764+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:39:40.428764+0000","last_peered":"2024-08-18T13:39:40.428764+0000","last_clean":"2024-08-18T13:39:40.428764+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:39:40.428764+0000","last_undegraded":"2024-08-18T13:39:40.428764+0000","last_fullsized":"2024-08-18T13:39:40.428764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423184+0000","last_change":"2024-08-18T13:39:35.400038+0000","last_active":"2024-08-18T13:39:39.423184+0000","last_peered":"2024-08-18T13:39:39.423184+0000","last_clean":"2024-08-18T13:39:39.423184+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:39:39.423184+0000","last_undegraded":"2024-08-18T13:39:39.423184+0000","last_fullsized":"2024-08-18T13:39:39.423184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437377+0000","last_change":"2024-08-18T13:39:39.436221+0000","last_active":"2024-08-18T13:39:39.437377+0000","last_peered":"2024-08-18T13:39:39.437377+0000","last_clean":"2024-08-18T13:39:39.437377+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:39:39.437377+0000","last_undegraded":"2024-08-18T13:39:39.437377+0000","last_fullsized":"2024-08-18T13:39:39.437377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428676+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.428676+0000","last_peered":"2024-08-18T13:39:39.428676+0000","last_clean":"2024-08-18T13:39:39.428676+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:39:39.428676+0000","last_undegraded":"2024-08-18T13:39:39.428676+0000","last_fullsized":"2024-08-18T13:39:39.428676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430090+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:39:40.430090+0000","last_peered":"2024-08-18T13:39:40.430090+0000","last_clean":"2024-08-18T13:39:40.430090+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:39:40.430090+0000","last_undegraded":"2024-08-18T13:39:40.430090+0000","last_fullsized":"2024-08-18T13:39:40.430090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.443080+0000","last_change":"2024-08-18T13:39:39.439633+0000","last_active":"2024-08-18T13:39:39.443080+0000","last_peered":"2024-08-18T13:39:39.443080+0000","last_clean":"2024-08-18T13:39:39.443080+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:39:39.443080+0000","last_undegraded":"2024-08-18T13:39:39.443080+0000","last_fullsized":"2024-08-18T13:39:39.443080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423607+0000","last_change":"2024-08-18T13:39:35.407947+0000","last_active":"2024-08-18T13:39:39.423607+0000","last_peered":"2024-08-18T13:39:39.423607+0000","last_clean":"2024-08-18T13:39:39.423607+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:39:39.423607+0000","last_undegraded":"2024-08-18T13:39:39.423607+0000","last_fullsized":"2024-08-18T13:39:39.423607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429156+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:39:39.429156+0000","last_peered":"2024-08-18T13:39:39.429156+0000","last_clean":"2024-08-18T13:39:39.429156+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:39:39.429156+0000","last_undegraded":"2024-08-18T13:39:39.429156+0000","last_fullsized":"2024-08-18T13:39:39.429156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442892+0000","last_change":"2024-08-18T13:39:39.439913+0000","last_active":"2024-08-18T13:39:39.442892+0000","last_peered":"2024-08-18T13:39:39.442892+0000","last_clean":"2024-08-18T13:39:39.442892+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:39:39.442892+0000","last_undegraded":"2024-08-18T13:39:39.442892+0000","last_fullsized":"2024-08-18T13:39:39.442892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430332+0000","last_change":"2024-08-18T13:39:35.403254+0000","last_active":"2024-08-18T13:39:39.430332+0000","last_peered":"2024-08-18T13:39:39.430332+0000","last_clean":"2024-08-18T13:39:39.430332+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:39:39.430332+0000","last_undegraded":"2024-08-18T13:39:39.430332+0000","last_fullsized":"2024-08-18T13:39:39.430332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422689+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:39:39.422689+0000","last_peered":"2024-08-18T13:39:39.422689+0000","last_clean":"2024-08-18T13:39:39.422689+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:39:39.422689+0000","last_undegraded":"2024-08-18T13:39:39.422689+0000","last_fullsized":"2024-08-18T13:39:39.422689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457822+0000","last_change":"2024-08-18T13:39:39.457250+0000","last_active":"2024-08-18T13:39:39.457822+0000","last_peered":"2024-08-18T13:39:39.457822+0000","last_clean":"2024-08-18T13:39:39.457822+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:39:39.457822+0000","last_undegraded":"2024-08-18T13:39:39.457822+0000","last_fullsized":"2024-08-18T13:39:39.457822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425047+0000","last_change":"2024-08-18T13:39:35.401612+0000","last_active":"2024-08-18T13:39:39.425047+0000","last_peered":"2024-08-18T13:39:39.425047+0000","last_clean":"2024-08-18T13:39:39.425047+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:39:39.425047+0000","last_undegraded":"2024-08-18T13:39:39.425047+0000","last_fullsized":"2024-08-18T13:39:39.425047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450234+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:39:39.450234+0000","last_peered":"2024-08-18T13:39:39.450234+0000","last_clean":"2024-08-18T13:39:39.450234+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:39:39.450234+0000","last_undegraded":"2024-08-18T13:39:39.450234+0000","last_fullsized":"2024-08-18T13:39:39.450234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447688+0000","last_change":"2024-08-18T13:39:39.443503+0000","last_active":"2024-08-18T13:39:39.447688+0000","last_peered":"2024-08-18T13:39:39.447688+0000","last_clean":"2024-08-18T13:39:39.447688+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:39:39.447688+0000","last_undegraded":"2024-08-18T13:39:39.447688+0000","last_fullsized":"2024-08-18T13:39:39.447688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432976+0000","last_change":"2024-08-18T13:39:35.407907+0000","last_active":"2024-08-18T13:39:39.432976+0000","last_peered":"2024-08-18T13:39:39.432976+0000","last_clean":"2024-08-18T13:39:39.432976+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:39:39.432976+0000","last_undegraded":"2024-08-18T13:39:39.432976+0000","last_fullsized":"2024-08-18T13:39:39.432976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429181+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:39:40.429181+0000","last_peered":"2024-08-18T13:39:40.429181+0000","last_clean":"2024-08-18T13:39:40.429181+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:39:40.429181+0000","last_undegraded":"2024-08-18T13:39:40.429181+0000","last_fullsized":"2024-08-18T13:39:40.429181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454767+0000","last_change":"2024-08-18T13:39:39.453138+0000","last_active":"2024-08-18T13:39:39.454767+0000","last_peered":"2024-08-18T13:39:39.454767+0000","last_clean":"2024-08-18T13:39:39.454767+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:39:39.454767+0000","last_undegraded":"2024-08-18T13:39:39.454767+0000","last_fullsized":"2024-08-18T13:39:39.454767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424552+0000","last_change":"2024-08-18T13:39:35.404453+0000","last_active":"2024-08-18T13:39:39.424552+0000","last_peered":"2024-08-18T13:39:39.424552+0000","last_clean":"2024-08-18T13:39:39.424552+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:39:39.424552+0000","last_undegraded":"2024-08-18T13:39:39.424552+0000","last_fullsized":"2024-08-18T13:39:39.424552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:41.051281+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:39:41.051281+0000","last_peered":"2024-08-18T13:39:41.051281+0000","last_clean":"2024-08-18T13:39:41.051281+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:39:41.051281+0000","last_undegraded":"2024-08-18T13:39:41.051281+0000","last_fullsized":"2024-08-18T13:39:41.051281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450645+0000","last_change":"2024-08-18T13:39:39.449851+0000","last_active":"2024-08-18T13:39:39.450645+0000","last_peered":"2024-08-18T13:39:39.450645+0000","last_clean":"2024-08-18T13:39:39.450645+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:39:39.450645+0000","last_undegraded":"2024-08-18T13:39:39.450645+0000","last_fullsized":"2024-08-18T13:39:39.450645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424459+0000","last_change":"2024-08-18T13:39:35.400604+0000","last_active":"2024-08-18T13:39:39.424459+0000","last_peered":"2024-08-18T13:39:39.424459+0000","last_clean":"2024-08-18T13:39:39.424459+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:39:39.424459+0000","last_undegraded":"2024-08-18T13:39:39.424459+0000","last_fullsized":"2024-08-18T13:39:39.424459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450871+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:39:39.450871+0000","last_peered":"2024-08-18T13:39:39.450871+0000","last_clean":"2024-08-18T13:39:39.450871+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:39:39.450871+0000","last_undegraded":"2024-08-18T13:39:39.450871+0000","last_fullsized":"2024-08-18T13:39:39.450871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454096+0000","last_change":"2024-08-18T13:39:39.453599+0000","last_active":"2024-08-18T13:39:39.454096+0000","last_peered":"2024-08-18T13:39:39.454096+0000","last_clean":"2024-08-18T13:39:39.454096+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:39:39.454096+0000","last_undegraded":"2024-08-18T13:39:39.454096+0000","last_fullsized":"2024-08-18T13:39:39.454096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423160+0000","last_change":"2024-08-18T13:39:35.410167+0000","last_active":"2024-08-18T13:39:39.423160+0000","last_peered":"2024-08-18T13:39:39.423160+0000","last_clean":"2024-08-18T13:39:39.423160+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:39:39.423160+0000","last_undegraded":"2024-08-18T13:39:39.423160+0000","last_fullsized":"2024-08-18T13:39:39.423160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424551+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:39:39.424551+0000","last_peered":"2024-08-18T13:39:39.424551+0000","last_clean":"2024-08-18T13:39:39.424551+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:39:39.424551+0000","last_undegraded":"2024-08-18T13:39:39.424551+0000","last_fullsized":"2024-08-18T13:39:39.424551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444517+0000","last_change":"2024-08-18T13:39:39.439948+0000","last_active":"2024-08-18T13:39:39.444517+0000","last_peered":"2024-08-18T13:39:39.444517+0000","last_clean":"2024-08-18T13:39:39.444517+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:39:39.444517+0000","last_undegraded":"2024-08-18T13:39:39.444517+0000","last_fullsized":"2024-08-18T13:39:39.444517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422894+0000","last_change":"2024-08-18T13:39:35.407829+0000","last_active":"2024-08-18T13:39:39.422894+0000","last_peered":"2024-08-18T13:39:39.422894+0000","last_clean":"2024-08-18T13:39:39.422894+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:39:39.422894+0000","last_undegraded":"2024-08-18T13:39:39.422894+0000","last_fullsized":"2024-08-18T13:39:39.422894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417877+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:39:39.417877+0000","last_peered":"2024-08-18T13:39:39.417877+0000","last_clean":"2024-08-18T13:39:39.417877+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:39:39.417877+0000","last_undegraded":"2024-08-18T13:39:39.417877+0000","last_fullsized":"2024-08-18T13:39:39.417877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445479+0000","last_change":"2024-08-18T13:39:39.442372+0000","last_active":"2024-08-18T13:39:39.445479+0000","last_peered":"2024-08-18T13:39:39.445479+0000","last_clean":"2024-08-18T13:39:39.445479+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:39:39.445479+0000","last_undegraded":"2024-08-18T13:39:39.445479+0000","last_fullsized":"2024-08-18T13:39:39.445479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421862+0000","last_change":"2024-08-18T13:39:35.400384+0000","last_active":"2024-08-18T13:39:39.421862+0000","last_peered":"2024-08-18T13:39:39.421862+0000","last_clean":"2024-08-18T13:39:39.421862+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:39:39.421862+0000","last_undegraded":"2024-08-18T13:39:39.421862+0000","last_fullsized":"2024-08-18T13:39:39.421862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415832+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:39:39.415832+0000","last_peered":"2024-08-18T13:39:39.415832+0000","last_clean":"2024-08-18T13:39:39.415832+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:39:39.415832+0000","last_undegraded":"2024-08-18T13:39:39.415832+0000","last_fullsized":"2024-08-18T13:39:39.415832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446859+0000","last_change":"2024-08-18T13:39:39.446401+0000","last_active":"2024-08-18T13:39:39.446859+0000","last_peered":"2024-08-18T13:39:39.446859+0000","last_clean":"2024-08-18T13:39:39.446859+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:39:39.446859+0000","last_undegraded":"2024-08-18T13:39:39.446859+0000","last_fullsized":"2024-08-18T13:39:39.446859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424307+0000","last_change":"2024-08-18T13:39:35.410127+0000","last_active":"2024-08-18T13:39:39.424307+0000","last_peered":"2024-08-18T13:39:39.424307+0000","last_clean":"2024-08-18T13:39:39.424307+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:39:39.424307+0000","last_undegraded":"2024-08-18T13:39:39.424307+0000","last_fullsized":"2024-08-18T13:39:39.424307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450338+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:39:39.450338+0000","last_peered":"2024-08-18T13:39:39.450338+0000","last_clean":"2024-08-18T13:39:39.450338+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:39:39.450338+0000","last_undegraded":"2024-08-18T13:39:39.450338+0000","last_fullsized":"2024-08-18T13:39:39.450338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454861+0000","last_change":"2024-08-18T13:39:39.453901+0000","last_active":"2024-08-18T13:39:39.454861+0000","last_peered":"2024-08-18T13:39:39.454861+0000","last_clean":"2024-08-18T13:39:39.454861+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:39:39.454861+0000","last_undegraded":"2024-08-18T13:39:39.454861+0000","last_fullsized":"2024-08-18T13:39:39.454861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430338+0000","last_change":"2024-08-18T13:39:35.412156+0000","last_active":"2024-08-18T13:39:39.430338+0000","last_peered":"2024-08-18T13:39:39.430338+0000","last_clean":"2024-08-18T13:39:39.430338+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:39:39.430338+0000","last_undegraded":"2024-08-18T13:39:39.430338+0000","last_fullsized":"2024-08-18T13:39:39.430338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417339+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:39:39.417339+0000","last_peered":"2024-08-18T13:39:39.417339+0000","last_clean":"2024-08-18T13:39:39.417339+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:39:39.417339+0000","last_undegraded":"2024-08-18T13:39:39.417339+0000","last_fullsized":"2024-08-18T13:39:39.417339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450727+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:39:39.450727+0000","last_peered":"2024-08-18T13:39:39.450727+0000","last_clean":"2024-08-18T13:39:39.450727+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:39:39.450727+0000","last_undegraded":"2024-08-18T13:39:39.450727+0000","last_fullsized":"2024-08-18T13:39:39.450727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454332+0000","last_change":"2024-08-18T13:39:39.453360+0000","last_active":"2024-08-18T13:39:39.454332+0000","last_peered":"2024-08-18T13:39:39.454332+0000","last_clean":"2024-08-18T13:39:39.454332+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:39:39.454332+0000","last_undegraded":"2024-08-18T13:39:39.454332+0000","last_fullsized":"2024-08-18T13:39:39.454332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450556+0000","last_change":"2024-08-18T13:39:35.421765+0000","last_active":"2024-08-18T13:39:39.450556+0000","last_peered":"2024-08-18T13:39:39.450556+0000","last_clean":"2024-08-18T13:39:39.450556+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:39:39.450556+0000","last_undegraded":"2024-08-18T13:39:39.450556+0000","last_fullsized":"2024-08-18T13:39:39.450556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416404+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:39:39.416404+0000","last_peered":"2024-08-18T13:39:39.416404+0000","last_clean":"2024-08-18T13:39:39.416404+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:39:39.416404+0000","last_undegraded":"2024-08-18T13:39:39.416404+0000","last_fullsized":"2024-08-18T13:39:39.416404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450222+0000","last_change":"2024-08-18T13:39:39.449653+0000","last_active":"2024-08-18T13:39:39.450222+0000","last_peered":"2024-08-18T13:39:39.450222+0000","last_clean":"2024-08-18T13:39:39.450222+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:39:39.450222+0000","last_undegraded":"2024-08-18T13:39:39.450222+0000","last_fullsized":"2024-08-18T13:39:39.450222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422710+0000","last_change":"2024-08-18T13:39:35.407439+0000","last_active":"2024-08-18T13:39:39.422710+0000","last_peered":"2024-08-18T13:39:39.422710+0000","last_clean":"2024-08-18T13:39:39.422710+0000","last_became_active":"2024-08-18T13:39:35.407128+0000","last_became_peered":"2024-08-18T13:39:35.407128+0000","last_unstale":"2024-08-18T13:39:39.422710+0000","last_undegraded":"2024-08-18T13:39:39.422710+0000","last_fullsized":"2024-08-18T13:39:39.422710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431851+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:39:40.431851+0000","last_peered":"2024-08-18T13:39:40.431851+0000","last_clean":"2024-08-18T13:39:40.431851+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:39:40.431851+0000","last_undegraded":"2024-08-18T13:39:40.431851+0000","last_fullsized":"2024-08-18T13:39:40.431851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445844+0000","last_change":"2024-08-18T13:39:39.443848+0000","last_active":"2024-08-18T13:39:39.445844+0000","last_peered":"2024-08-18T13:39:39.445844+0000","last_clean":"2024-08-18T13:39:39.445844+0000","last_became_active":"2024-08-18T13:39:39.443456+0000","last_became_peered":"2024-08-18T13:39:39.443456+0000","last_unstale":"2024-08-18T13:39:39.445844+0000","last_undegraded":"2024-08-18T13:39:39.445844+0000","last_fullsized":"2024-08-18T13:39:39.445844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:27.994932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450757+0000","last_change":"2024-08-18T13:39:35.421612+0000","last_active":"2024-08-18T13:39:39.450757+0000","last_peered":"2024-08-18T13:39:39.450757+0000","last_clean":"2024-08-18T13:39:39.450757+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:39:39.450757+0000","last_undegraded":"2024-08-18T13:39:39.450757+0000","last_fullsized":"2024-08-18T13:39:39.450757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433442+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:39:40.433442+0000","last_peered":"2024-08-18T13:39:40.433442+0000","last_clean":"2024-08-18T13:39:40.433442+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:39:40.433442+0000","last_undegraded":"2024-08-18T13:39:40.433442+0000","last_fullsized":"2024-08-18T13:39:40.433442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454815+0000","last_change":"2024-08-18T13:39:39.453520+0000","last_active":"2024-08-18T13:39:39.454815+0000","last_peered":"2024-08-18T13:39:39.454815+0000","last_clean":"2024-08-18T13:39:39.454815+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:39:39.454815+0000","last_undegraded":"2024-08-18T13:39:39.454815+0000","last_fullsized":"2024-08-18T13:39:39.454815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450976+0000","last_change":"2024-08-18T13:39:35.421465+0000","last_active":"2024-08-18T13:39:39.450976+0000","last_peered":"2024-08-18T13:39:39.450976+0000","last_clean":"2024-08-18T13:39:39.450976+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:39:39.450976+0000","last_undegraded":"2024-08-18T13:39:39.450976+0000","last_fullsized":"2024-08-18T13:39:39.450976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429394+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:39:40.429394+0000","last_peered":"2024-08-18T13:39:40.429394+0000","last_clean":"2024-08-18T13:39:40.429394+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:39:40.429394+0000","last_undegraded":"2024-08-18T13:39:40.429394+0000","last_fullsized":"2024-08-18T13:39:40.429394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.448067+0000","last_change":"2024-08-18T13:39:39.446029+0000","last_active":"2024-08-18T13:39:39.448067+0000","last_peered":"2024-08-18T13:39:39.448067+0000","last_clean":"2024-08-18T13:39:39.448067+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:39:39.448067+0000","last_undegraded":"2024-08-18T13:39:39.448067+0000","last_fullsized":"2024-08-18T13:39:39.448067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450468+0000","last_change":"2024-08-18T13:39:35.417094+0000","last_active":"2024-08-18T13:39:39.450468+0000","last_peered":"2024-08-18T13:39:39.450468+0000","last_clean":"2024-08-18T13:39:39.450468+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:39:39.450468+0000","last_undegraded":"2024-08-18T13:39:39.450468+0000","last_fullsized":"2024-08-18T13:39:39.450468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.436721+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:39:40.436721+0000","last_peered":"2024-08-18T13:39:40.436721+0000","last_clean":"2024-08-18T13:39:40.436721+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:39:40.436721+0000","last_undegraded":"2024-08-18T13:39:40.436721+0000","last_fullsized":"2024-08-18T13:39:40.436721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437633+0000","last_change":"2024-08-18T13:39:39.436992+0000","last_active":"2024-08-18T13:39:39.437633+0000","last_peered":"2024-08-18T13:39:39.437633+0000","last_clean":"2024-08-18T13:39:39.437633+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:39:39.437633+0000","last_undegraded":"2024-08-18T13:39:39.437633+0000","last_fullsized":"2024-08-18T13:39:39.437633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427745+0000","last_change":"2024-08-18T13:39:35.411315+0000","last_active":"2024-08-18T13:39:39.427745+0000","last_peered":"2024-08-18T13:39:39.427745+0000","last_clean":"2024-08-18T13:39:39.427745+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:39:39.427745+0000","last_undegraded":"2024-08-18T13:39:39.427745+0000","last_fullsized":"2024-08-18T13:39:39.427745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428272+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:39:39.428272+0000","last_peered":"2024-08-18T13:39:39.428272+0000","last_clean":"2024-08-18T13:39:39.428272+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:39:39.428272+0000","last_undegraded":"2024-08-18T13:39:39.428272+0000","last_fullsized":"2024-08-18T13:39:39.428272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.458299+0000","last_change":"2024-08-18T13:39:39.456955+0000","last_active":"2024-08-18T13:39:39.458299+0000","last_peered":"2024-08-18T13:39:39.458299+0000","last_clean":"2024-08-18T13:39:39.458299+0000","last_became_active":"2024-08-18T13:39:39.456534+0000","last_became_peered":"2024-08-18T13:39:39.456534+0000","last_unstale":"2024-08-18T13:39:39.458299+0000","last_undegraded":"2024-08-18T13:39:39.458299+0000","last_fullsized":"2024-08-18T13:39:39.458299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:26:42.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450840+0000","last_change":"2024-08-18T13:39:35.417348+0000","last_active":"2024-08-18T13:39:39.450840+0000","last_peered":"2024-08-18T13:39:39.450840+0000","last_clean":"2024-08-18T13:39:39.450840+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:39:39.450840+0000","last_undegraded":"2024-08-18T13:39:39.450840+0000","last_fullsized":"2024-08-18T13:39:39.450840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417725+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:39:39.417725+0000","last_peered":"2024-08-18T13:39:39.417725+0000","last_clean":"2024-08-18T13:39:39.417725+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:39:39.417725+0000","last_undegraded":"2024-08-18T13:39:39.417725+0000","last_fullsized":"2024-08-18T13:39:39.417725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454724+0000","last_change":"2024-08-18T13:39:39.453682+0000","last_active":"2024-08-18T13:39:39.454724+0000","last_peered":"2024-08-18T13:39:39.454724+0000","last_clean":"2024-08-18T13:39:39.454724+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:39:39.454724+0000","last_undegraded":"2024-08-18T13:39:39.454724+0000","last_fullsized":"2024-08-18T13:39:39.454724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427706+0000","last_change":"2024-08-18T13:39:35.407127+0000","last_active":"2024-08-18T13:39:39.427706+0000","last_peered":"2024-08-18T13:39:39.427706+0000","last_clean":"2024-08-18T13:39:39.427706+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:39:39.427706+0000","last_undegraded":"2024-08-18T13:39:39.427706+0000","last_fullsized":"2024-08-18T13:39:39.427706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431523+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:39:40.431523+0000","last_peered":"2024-08-18T13:39:40.431523+0000","last_clean":"2024-08-18T13:39:40.431523+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:39:40.431523+0000","last_undegraded":"2024-08-18T13:39:40.431523+0000","last_fullsized":"2024-08-18T13:39:40.431523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457313+0000","last_change":"2024-08-18T13:39:39.456626+0000","last_active":"2024-08-18T13:39:39.457313+0000","last_peered":"2024-08-18T13:39:39.457313+0000","last_clean":"2024-08-18T13:39:39.457313+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:39:39.457313+0000","last_undegraded":"2024-08-18T13:39:39.457313+0000","last_fullsized":"2024-08-18T13:39:39.457313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450395+0000","last_change":"2024-08-18T13:39:35.417693+0000","last_active":"2024-08-18T13:39:39.450395+0000","last_peered":"2024-08-18T13:39:39.450395+0000","last_clean":"2024-08-18T13:39:39.450395+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:39:39.450395+0000","last_undegraded":"2024-08-18T13:39:39.450395+0000","last_fullsized":"2024-08-18T13:39:39.450395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:48.063001+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:39:48.063001+0000","last_peered":"2024-08-18T13:39:48.063001+0000","last_clean":"2024-08-18T13:39:48.063001+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:39:48.063001+0000","last_undegraded":"2024-08-18T13:39:48.063001+0000","last_fullsized":"2024-08-18T13:39:48.063001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3762,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.440639+0000","last_change":"2024-08-18T13:39:39.438064+0000","last_active":"2024-08-18T13:39:39.440639+0000","last_peered":"2024-08-18T13:39:39.440639+0000","last_clean":"2024-08-18T13:39:39.440639+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:39:39.440639+0000","last_undegraded":"2024-08-18T13:39:39.440639+0000","last_fullsized":"2024-08-18T13:39:39.440639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421839+0000","last_change":"2024-08-18T13:39:35.406398+0000","last_active":"2024-08-18T13:39:39.421839+0000","last_peered":"2024-08-18T13:39:39.421839+0000","last_clean":"2024-08-18T13:39:39.421839+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:39:39.421839+0000","last_undegraded":"2024-08-18T13:39:39.421839+0000","last_fullsized":"2024-08-18T13:39:39.421839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433257+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:39:40.433257+0000","last_peered":"2024-08-18T13:39:40.433257+0000","last_clean":"2024-08-18T13:39:40.433257+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:39:40.433257+0000","last_undegraded":"2024-08-18T13:39:40.433257+0000","last_fullsized":"2024-08-18T13:39:40.433257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444385+0000","last_change":"2024-08-18T13:39:39.439962+0000","last_active":"2024-08-18T13:39:39.444385+0000","last_peered":"2024-08-18T13:39:39.444385+0000","last_clean":"2024-08-18T13:39:39.444385+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:39:39.444385+0000","last_undegraded":"2024-08-18T13:39:39.444385+0000","last_fullsized":"2024-08-18T13:39:39.444385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432863+0000","last_change":"2024-08-18T13:39:35.411067+0000","last_active":"2024-08-18T13:39:39.432863+0000","last_peered":"2024-08-18T13:39:39.432863+0000","last_clean":"2024-08-18T13:39:39.432863+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:39:39.432863+0000","last_undegraded":"2024-08-18T13:39:39.432863+0000","last_fullsized":"2024-08-18T13:39:39.432863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422811+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:39:39.422811+0000","last_peered":"2024-08-18T13:39:39.422811+0000","last_clean":"2024-08-18T13:39:39.422811+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:39:39.422811+0000","last_undegraded":"2024-08-18T13:39:39.422811+0000","last_fullsized":"2024-08-18T13:39:39.422811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447069+0000","last_change":"2024-08-18T13:39:39.443938+0000","last_active":"2024-08-18T13:39:39.447069+0000","last_peered":"2024-08-18T13:39:39.447069+0000","last_clean":"2024-08-18T13:39:39.447069+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:39:39.447069+0000","last_undegraded":"2024-08-18T13:39:39.447069+0000","last_fullsized":"2024-08-18T13:39:39.447069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422769+0000","last_change":"2024-08-18T13:39:35.409517+0000","last_active":"2024-08-18T13:39:39.422769+0000","last_peered":"2024-08-18T13:39:39.422769+0000","last_clean":"2024-08-18T13:39:39.422769+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:39:39.422769+0000","last_undegraded":"2024-08-18T13:39:39.422769+0000","last_fullsized":"2024-08-18T13:39:39.422769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429841+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:39:39.429841+0000","last_peered":"2024-08-18T13:39:39.429841+0000","last_clean":"2024-08-18T13:39:39.429841+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:39:39.429841+0000","last_undegraded":"2024-08-18T13:39:39.429841+0000","last_fullsized":"2024-08-18T13:39:39.429841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450637+0000","last_change":"2024-08-18T13:39:39.449671+0000","last_active":"2024-08-18T13:39:39.450637+0000","last_peered":"2024-08-18T13:39:39.450637+0000","last_clean":"2024-08-18T13:39:39.450637+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:39:39.450637+0000","last_undegraded":"2024-08-18T13:39:39.450637+0000","last_fullsized":"2024-08-18T13:39:39.450637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454824+0000","last_change":"2024-08-18T13:39:39.453907+0000","last_active":"2024-08-18T13:39:39.454824+0000","last_peered":"2024-08-18T13:39:39.454824+0000","last_clean":"2024-08-18T13:39:39.454824+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:39:39.454824+0000","last_undegraded":"2024-08-18T13:39:39.454824+0000","last_fullsized":"2024-08-18T13:39:39.454824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422264+0000","last_change":"2024-08-18T13:39:35.401997+0000","last_active":"2024-08-18T13:39:39.422264+0000","last_peered":"2024-08-18T13:39:39.422264+0000","last_clean":"2024-08-18T13:39:39.422264+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:39:39.422264+0000","last_undegraded":"2024-08-18T13:39:39.422264+0000","last_fullsized":"2024-08-18T13:39:39.422264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418448+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:39:39.418448+0000","last_peered":"2024-08-18T13:39:39.418448+0000","last_clean":"2024-08-18T13:39:39.418448+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:39:39.418448+0000","last_undegraded":"2024-08-18T13:39:39.418448+0000","last_fullsized":"2024-08-18T13:39:39.418448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442977+0000","last_change":"2024-08-18T13:39:39.440811+0000","last_active":"2024-08-18T13:39:39.442977+0000","last_peered":"2024-08-18T13:39:39.442977+0000","last_clean":"2024-08-18T13:39:39.442977+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:39:39.442977+0000","last_undegraded":"2024-08-18T13:39:39.442977+0000","last_fullsized":"2024-08-18T13:39:39.442977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430210+0000","last_change":"2024-08-18T13:39:35.412127+0000","last_active":"2024-08-18T13:39:39.430210+0000","last_peered":"2024-08-18T13:39:39.430210+0000","last_clean":"2024-08-18T13:39:39.430210+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:39:39.430210+0000","last_undegraded":"2024-08-18T13:39:39.430210+0000","last_fullsized":"2024-08-18T13:39:39.430210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423847+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:39:39.423847+0000","last_peered":"2024-08-18T13:39:39.423847+0000","last_clean":"2024-08-18T13:39:39.423847+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:39:39.423847+0000","last_undegraded":"2024-08-18T13:39:39.423847+0000","last_fullsized":"2024-08-18T13:39:39.423847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429678+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:39:40.429678+0000","last_peered":"2024-08-18T13:39:40.429678+0000","last_clean":"2024-08-18T13:39:40.429678+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:39:40.429678+0000","last_undegraded":"2024-08-18T13:39:40.429678+0000","last_fullsized":"2024-08-18T13:39:40.429678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429574+0000","last_change":"2024-08-18T13:39:35.407995+0000","last_active":"2024-08-18T13:39:39.429574+0000","last_peered":"2024-08-18T13:39:39.429574+0000","last_clean":"2024-08-18T13:39:39.429574+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:39:39.429574+0000","last_undegraded":"2024-08-18T13:39:39.429574+0000","last_fullsized":"2024-08-18T13:39:39.429574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.438294+0000","last_change":"2024-08-18T13:39:39.437542+0000","last_active":"2024-08-18T13:39:39.438294+0000","last_peered":"2024-08-18T13:39:39.438294+0000","last_clean":"2024-08-18T13:39:39.438294+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:39:39.438294+0000","last_undegraded":"2024-08-18T13:39:39.438294+0000","last_fullsized":"2024-08-18T13:39:39.438294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434310+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:39:40.434310+0000","last_peered":"2024-08-18T13:39:40.434310+0000","last_clean":"2024-08-18T13:39:40.434310+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:39:40.434310+0000","last_undegraded":"2024-08-18T13:39:40.434310+0000","last_fullsized":"2024-08-18T13:39:40.434310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422760+0000","last_change":"2024-08-18T13:39:35.403759+0000","last_active":"2024-08-18T13:39:39.422760+0000","last_peered":"2024-08-18T13:39:39.422760+0000","last_clean":"2024-08-18T13:39:39.422760+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:39:39.422760+0000","last_undegraded":"2024-08-18T13:39:39.422760+0000","last_fullsized":"2024-08-18T13:39:39.422760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446432+0000","last_change":"2024-08-18T13:39:39.441201+0000","last_active":"2024-08-18T13:39:39.446432+0000","last_peered":"2024-08-18T13:39:39.446432+0000","last_clean":"2024-08-18T13:39:39.446432+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:39:39.446432+0000","last_undegraded":"2024-08-18T13:39:39.446432+0000","last_fullsized":"2024-08-18T13:39:39.446432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427616+0000","last_change":"2024-08-18T13:39:35.413292+0000","last_active":"2024-08-18T13:39:39.427616+0000","last_peered":"2024-08-18T13:39:39.427616+0000","last_clean":"2024-08-18T13:39:39.427616+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:39:39.427616+0000","last_undegraded":"2024-08-18T13:39:39.427616+0000","last_fullsized":"2024-08-18T13:39:39.427616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415812+0000","last_change":"2024-08-18T13:39:35.401196+0000","last_active":"2024-08-18T13:39:39.415812+0000","last_peered":"2024-08-18T13:39:39.415812+0000","last_clean":"2024-08-18T13:39:39.415812+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:39:39.415812+0000","last_undegraded":"2024-08-18T13:39:39.415812+0000","last_fullsized":"2024-08-18T13:39:39.415812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417577+0000","last_change":"2024-08-18T13:39:35.406603+0000","last_active":"2024-08-18T13:39:39.417577+0000","last_peered":"2024-08-18T13:39:39.417577+0000","last_clean":"2024-08-18T13:39:39.417577+0000","last_became_active":"2024-08-18T13:39:35.406331+0000","last_became_peered":"2024-08-18T13:39:35.406331+0000","last_unstale":"2024-08-18T13:39:39.417577+0000","last_undegraded":"2024-08-18T13:39:39.417577+0000","last_fullsized":"2024-08-18T13:39:39.417577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:21:12.232172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429494+0000","last_change":"2024-08-18T13:39:35.404051+0000","last_active":"2024-08-18T13:39:39.429494+0000","last_peered":"2024-08-18T13:39:39.429494+0000","last_clean":"2024-08-18T13:39:39.429494+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:39:39.429494+0000","last_undegraded":"2024-08-18T13:39:39.429494+0000","last_fullsized":"2024-08-18T13:39:39.429494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422959+0000","last_change":"2024-08-18T13:39:35.409288+0000","last_active":"2024-08-18T13:39:39.422959+0000","last_peered":"2024-08-18T13:39:39.422959+0000","last_clean":"2024-08-18T13:39:39.422959+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:39:39.422959+0000","last_undegraded":"2024-08-18T13:39:39.422959+0000","last_fullsized":"2024-08-18T13:39:39.422959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424121+0000","last_change":"2024-08-18T13:39:35.403051+0000","last_active":"2024-08-18T13:39:39.424121+0000","last_peered":"2024-08-18T13:39:39.424121+0000","last_clean":"2024-08-18T13:39:39.424121+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:39:39.424121+0000","last_undegraded":"2024-08-18T13:39:39.424121+0000","last_fullsized":"2024-08-18T13:39:39.424121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430049+0000","last_change":"2024-08-18T13:39:35.412281+0000","last_active":"2024-08-18T13:39:39.430049+0000","last_peered":"2024-08-18T13:39:39.430049+0000","last_clean":"2024-08-18T13:39:39.430049+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:39:39.430049+0000","last_undegraded":"2024-08-18T13:39:39.430049+0000","last_fullsized":"2024-08-18T13:39:39.430049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424466+0000","last_change":"2024-08-18T13:39:35.409220+0000","last_active":"2024-08-18T13:39:39.424466+0000","last_peered":"2024-08-18T13:39:39.424466+0000","last_clean":"2024-08-18T13:39:39.424466+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:39:39.424466+0000","last_undegraded":"2024-08-18T13:39:39.424466+0000","last_fullsized":"2024-08-18T13:39:39.424466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418253+0000","last_change":"2024-08-18T13:39:35.401759+0000","last_active":"2024-08-18T13:39:39.418253+0000","last_peered":"2024-08-18T13:39:39.418253+0000","last_clean":"2024-08-18T13:39:39.418253+0000","last_became_active":"2024-08-18T13:39:35.401476+0000","last_became_peered":"2024-08-18T13:39:35.401476+0000","last_unstale":"2024-08-18T13:39:39.418253+0000","last_undegraded":"2024-08-18T13:39:39.418253+0000","last_fullsized":"2024-08-18T13:39:39.418253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:04:43.997597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423669+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.423669+0000","last_peered":"2024-08-18T13:39:39.423669+0000","last_clean":"2024-08-18T13:39:39.423669+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:39:39.423669+0000","last_undegraded":"2024-08-18T13:39:39.423669+0000","last_fullsized":"2024-08-18T13:39:39.423669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417410+0000","last_change":"2024-08-18T13:39:35.407218+0000","last_active":"2024-08-18T13:39:39.417410+0000","last_peered":"2024-08-18T13:39:39.417410+0000","last_clean":"2024-08-18T13:39:39.417410+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:39:39.417410+0000","last_undegraded":"2024-08-18T13:39:39.417410+0000","last_fullsized":"2024-08-18T13:39:39.417410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429387+0000","last_change":"2024-08-18T13:39:35.407339+0000","last_active":"2024-08-18T13:39:39.429387+0000","last_peered":"2024-08-18T13:39:39.429387+0000","last_clean":"2024-08-18T13:39:39.429387+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:39:39.429387+0000","last_undegraded":"2024-08-18T13:39:39.429387+0000","last_fullsized":"2024-08-18T13:39:39.429387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429062+0000","last_change":"2024-08-18T13:39:35.408647+0000","last_active":"2024-08-18T13:39:39.429062+0000","last_peered":"2024-08-18T13:39:39.429062+0000","last_clean":"2024-08-18T13:39:39.429062+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:39:39.429062+0000","last_undegraded":"2024-08-18T13:39:39.429062+0000","last_fullsized":"2024-08-18T13:39:39.429062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450837+0000","last_change":"2024-08-18T13:39:35.418000+0000","last_active":"2024-08-18T13:39:39.450837+0000","last_peered":"2024-08-18T13:39:39.450837+0000","last_clean":"2024-08-18T13:39:39.450837+0000","last_became_active":"2024-08-18T13:39:35.417661+0000","last_became_peered":"2024-08-18T13:39:35.417661+0000","last_unstale":"2024-08-18T13:39:39.450837+0000","last_undegraded":"2024-08-18T13:39:39.450837+0000","last_fullsized":"2024-08-18T13:39:39.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:53:44.739585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416124+0000","last_change":"2024-08-18T13:39:35.405425+0000","last_active":"2024-08-18T13:39:39.416124+0000","last_peered":"2024-08-18T13:39:39.416124+0000","last_clean":"2024-08-18T13:39:39.416124+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:39:39.416124+0000","last_undegraded":"2024-08-18T13:39:39.416124+0000","last_fullsized":"2024-08-18T13:39:39.416124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423886+0000","last_change":"2024-08-18T13:39:35.400889+0000","last_active":"2024-08-18T13:39:39.423886+0000","last_peered":"2024-08-18T13:39:39.423886+0000","last_clean":"2024-08-18T13:39:39.423886+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:39:39.423886+0000","last_undegraded":"2024-08-18T13:39:39.423886+0000","last_fullsized":"2024-08-18T13:39:39.423886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418057+0000","last_change":"2024-08-18T13:39:35.407634+0000","last_active":"2024-08-18T13:39:39.418057+0000","last_peered":"2024-08-18T13:39:39.418057+0000","last_clean":"2024-08-18T13:39:39.418057+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:39:39.418057+0000","last_undegraded":"2024-08-18T13:39:39.418057+0000","last_fullsized":"2024-08-18T13:39:39.418057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422473+0000","last_change":"2024-08-18T13:39:35.397509+0000","last_active":"2024-08-18T13:39:39.422473+0000","last_peered":"2024-08-18T13:39:39.422473+0000","last_clean":"2024-08-18T13:39:39.422473+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:39:39.422473+0000","last_undegraded":"2024-08-18T13:39:39.422473+0000","last_fullsized":"2024-08-18T13:39:39.422473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425330+0000","last_change":"2024-08-18T13:39:35.404533+0000","last_active":"2024-08-18T13:39:39.425330+0000","last_peered":"2024-08-18T13:39:39.425330+0000","last_clean":"2024-08-18T13:39:39.425330+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:39:39.425330+0000","last_undegraded":"2024-08-18T13:39:39.425330+0000","last_fullsized":"2024-08-18T13:39:39.425330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425567+0000","last_change":"2024-08-18T13:39:35.404839+0000","last_active":"2024-08-18T13:39:39.425567+0000","last_peered":"2024-08-18T13:39:39.425567+0000","last_clean":"2024-08-18T13:39:39.425567+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:39:39.425567+0000","last_undegraded":"2024-08-18T13:39:39.425567+0000","last_fullsized":"2024-08-18T13:39:39.425567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421633+0000","last_change":"2024-08-18T13:39:35.405867+0000","last_active":"2024-08-18T13:39:39.421633+0000","last_peered":"2024-08-18T13:39:39.421633+0000","last_clean":"2024-08-18T13:39:39.421633+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:39:39.421633+0000","last_undegraded":"2024-08-18T13:39:39.421633+0000","last_fullsized":"2024-08-18T13:39:39.421633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429697+0000","last_change":"2024-08-18T13:39:35.411173+0000","last_active":"2024-08-18T13:39:39.429697+0000","last_peered":"2024-08-18T13:39:39.429697+0000","last_clean":"2024-08-18T13:39:39.429697+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:39:39.429697+0000","last_undegraded":"2024-08-18T13:39:39.429697+0000","last_fullsized":"2024-08-18T13:39:39.429697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429962+0000","last_change":"2024-08-18T13:39:35.418013+0000","last_active":"2024-08-18T13:39:39.429962+0000","last_peered":"2024-08-18T13:39:39.429962+0000","last_clean":"2024-08-18T13:39:39.429962+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:39:39.429962+0000","last_undegraded":"2024-08-18T13:39:39.429962+0000","last_fullsized":"2024-08-18T13:39:39.429962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450803+0000","last_change":"2024-08-18T13:39:35.421633+0000","last_active":"2024-08-18T13:39:39.450803+0000","last_peered":"2024-08-18T13:39:39.450803+0000","last_clean":"2024-08-18T13:39:39.450803+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:39:39.450803+0000","last_undegraded":"2024-08-18T13:39:39.450803+0000","last_fullsized":"2024-08-18T13:39:39.450803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425234+0000","last_change":"2024-08-18T13:39:35.411443+0000","last_active":"2024-08-18T13:39:39.425234+0000","last_peered":"2024-08-18T13:39:39.425234+0000","last_clean":"2024-08-18T13:39:39.425234+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:39:39.425234+0000","last_undegraded":"2024-08-18T13:39:39.425234+0000","last_fullsized":"2024-08-18T13:39:39.425234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450366+0000","last_change":"2024-08-18T13:39:35.415548+0000","last_active":"2024-08-18T13:39:39.450366+0000","last_peered":"2024-08-18T13:39:39.450366+0000","last_clean":"2024-08-18T13:39:39.450366+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:39:39.450366+0000","last_undegraded":"2024-08-18T13:39:39.450366+0000","last_fullsized":"2024-08-18T13:39:39.450366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417246+0000","last_change":"2024-08-18T13:39:35.410795+0000","last_active":"2024-08-18T13:39:39.417246+0000","last_peered":"2024-08-18T13:39:39.417246+0000","last_clean":"2024-08-18T13:39:39.417246+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:39:39.417246+0000","last_undegraded":"2024-08-18T13:39:39.417246+0000","last_fullsized":"2024-08-18T13:39:39.417246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450710+0000","last_change":"2024-08-18T13:39:35.417550+0000","last_active":"2024-08-18T13:39:39.450710+0000","last_peered":"2024-08-18T13:39:39.450710+0000","last_clean":"2024-08-18T13:39:39.450710+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:39:39.450710+0000","last_undegraded":"2024-08-18T13:39:39.450710+0000","last_fullsized":"2024-08-18T13:39:39.450710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422555+0000","last_change":"2024-08-18T13:39:35.410002+0000","last_active":"2024-08-18T13:39:39.422555+0000","last_peered":"2024-08-18T13:39:39.422555+0000","last_clean":"2024-08-18T13:39:39.422555+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:39:39.422555+0000","last_undegraded":"2024-08-18T13:39:39.422555+0000","last_fullsized":"2024-08-18T13:39:39.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427751+0000","last_change":"2024-08-18T13:39:35.406627+0000","last_active":"2024-08-18T13:39:39.427751+0000","last_peered":"2024-08-18T13:39:39.427751+0000","last_clean":"2024-08-18T13:39:39.427751+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:39:39.427751+0000","last_undegraded":"2024-08-18T13:39:39.427751+0000","last_fullsized":"2024-08-18T13:39:39.427751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423078+0000","last_change":"2024-08-18T13:39:35.405750+0000","last_active":"2024-08-18T13:39:39.423078+0000","last_peered":"2024-08-18T13:39:39.423078+0000","last_clean":"2024-08-18T13:39:39.423078+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:39:39.423078+0000","last_undegraded":"2024-08-18T13:39:39.423078+0000","last_fullsized":"2024-08-18T13:39:39.423078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450591+0000","last_change":"2024-08-18T13:39:35.417195+0000","last_active":"2024-08-18T13:39:39.450591+0000","last_peered":"2024-08-18T13:39:39.450591+0000","last_clean":"2024-08-18T13:39:39.450591+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:39:39.450591+0000","last_undegraded":"2024-08-18T13:39:39.450591+0000","last_fullsized":"2024-08-18T13:39:39.450591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4548,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":856064,"data_stored":693001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]}]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]}]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":270336,"data_stored":102189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]}]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":105825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":258048,"data_stored":102063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":102223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":864256,"data_stored":693057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":262144,"data_stored":102153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:40:02.791 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-18T13:40:02.791 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-18T13:40:02.791 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-18T13:40:02.791 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-18T13:40:02.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdea2b700 1 Processor -- start 2024-08-18T13:40:02.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdea2b700 1 -- start start 2024-08-18T13:40:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdea2b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5bd814f1e0 0x7f5bd80c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdea2b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd80ac450 con 0x7f5bd80c91e0 2024-08-18T13:40:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdea2b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd80c8210 con 0x7f5bd8155be0 2024-08-18T13:40:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdea2b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd80ab8c0 con 0x7f5bd814f1e0 2024-08-18T13:40:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdda29700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5bd814f1e0 0x7f5bd80c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdda29700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5bd814f1e0 0x7f5bd80c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45452/0 (socket says 172.21.15.118:45452) 2024-08-18T13:40:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdda29700 1 -- 172.21.15.118:0/769443181 learned_addr learned my addr 172.21.15.118:0/769443181 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdda29700 1 -- 172.21.15.118:0/769443181 >> v1:172.21.15.178:6789/0 conn(0x7f5bd8155be0 legacy=0x7f5bd80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdda29700 1 -- 172.21.15.118:0/769443181 >> v1:172.21.15.118:6789/0 conn(0x7f5bd80c91e0 legacy=0x7f5bd80c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:02.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.886+0000 7f5bdda29700 1 -- 172.21.15.118:0/769443181 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd80abb60 con 0x7f5bd814f1e0 2024-08-18T13:40:02.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdda29700 1 --2- 172.21.15.118:0/769443181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd814f1e0 0x7f5bd80c2640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009f20 tx=0x7f5bd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244a1524e63c7292 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdca27700 1 -- 172.21.15.118:0/769443181 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000e070 con 0x7f5bd814f1e0 2024-08-18T13:40:02.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdca27700 1 -- 172.21.15.118:0/769443181 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000bad0 con 0x7f5bd814f1e0 2024-08-18T13:40:02.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdca27700 1 -- 172.21.15.118:0/769443181 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00092e0 con 0x7f5bd814f1e0 2024-08-18T13:40:02.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdea2b700 1 -- 172.21.15.118:0/769443181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd814f1e0 msgr2=0x7f5bd80c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/769443181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd814f1e0 0x7f5bd80c2640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009f20 tx=0x7f5bd0009840 comp rx=0 tx=0).stop 2024-08-18T13:40:02.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdea2b700 1 -- 172.21.15.118:0/769443181 shutdown_connections 2024-08-18T13:40:02.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/769443181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd814f1e0 0x7f5bd80c2640 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdea2b700 1 -- 172.21.15.118:0/769443181 >> 172.21.15.118:0/769443181 conn(0x7f5bd80c8620 msgr2=0x7f5bd80c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:02.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdea2b700 1 -- 172.21.15.118:0/769443181 shutdown_connections 2024-08-18T13:40:02.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.890+0000 7f5bdea2b700 1 -- 172.21.15.118:0/769443181 wait complete. 2024-08-18T13:40:02.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 Processor -- start 2024-08-18T13:40:02.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 -- start start 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd80c91e0 0x7f5bd8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd814f1e0 0x7f5bd8067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8067ab0 0x7f5bd8068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80b0660 con 0x7f5bd80c91e0 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8154370 con 0x7f5bd814f1e0 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdea2b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80b0850 con 0x7f5bd8067ab0 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd80c91e0 0x7f5bd8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd80c91e0 0x7f5bd8069f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45466/0 (socket says 172.21.15.118:45466) 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 -- 172.21.15.118:0/1764087424 learned_addr learned my addr 172.21.15.118:0/1764087424 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdd228700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd814f1e0 0x7f5bd8067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bde22a700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8067ab0 0x7f5bd8068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 -- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8067ab0 msgr2=0x7f5bd8068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8067ab0 0x7f5bd8068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 -- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd814f1e0 msgr2=0x7f5bd8067570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd814f1e0 0x7f5bd8067570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 -- 172.21.15.118:0/1764087424 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd0009ae0 con 0x7f5bd80c91e0 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bde22a700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8067ab0 0x7f5bd8068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bdda29700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd80c91e0 0x7f5bd8069f60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009430 tx=0x7f5bd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.894+0000 7f5bc9ffb700 1 -- 172.21.15.118:0/1764087424 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00094e0 con 0x7f5bd80c91e0 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.898+0000 7f5bc9ffb700 1 -- 172.21.15.118:0/1764087424 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd0012b30 con 0x7f5bd80c91e0 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.898+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd8166010 con 0x7f5bd80c91e0 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.898+0000 7f5bc9ffb700 1 -- 172.21.15.118:0/1764087424 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd001d420 con 0x7f5bd80c91e0 2024-08-18T13:40:02.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.898+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd8067fa0 con 0x7f5bd80c91e0 2024-08-18T13:40:02.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.898+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd80713a0 con 0x7f5bd80c91e0 2024-08-18T13:40:02.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.898+0000 7f5bc9ffb700 1 -- 172.21.15.118:0/1764087424 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5bd001d810 con 0x7f5bd80c91e0 2024-08-18T13:40:02.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.902+0000 7f5bc9ffb700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bbc060860 0x7f5bbc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:02.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.902+0000 7f5bc9ffb700 1 -- 172.21.15.118:0/1764087424 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f5bd007e600 con 0x7f5bd80c91e0 2024-08-18T13:40:02.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.902+0000 7f5bdd228700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bbc060860 0x7f5bbc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:02.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.902+0000 7f5bdd228700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bbc060860 0x7f5bbc062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bd806a670 tx=0x7f5bcc007ea0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:02.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:02.906+0000 7f5bc9ffb700 1 -- 172.21.15.118:0/1764087424 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5bd004e760 con 0x7f5bd80c91e0 2024-08-18T13:40:03.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.090+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5bd80c2420 con 0x7f5bd80c91e0 2024-08-18T13:40:03.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.090+0000 7f5bc9ffb700 1 -- 172.21.15.118:0/1764087424 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5bd0051d80 con 0x7f5bd80c91e0 2024-08-18T13:40:03.096 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:03.096 INFO:teuthology.orchestra.run.smithi118.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bbc060860 msgr2=0x7f5bbc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bbc060860 0x7f5bbc062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bd806a670 tx=0x7f5bcc007ea0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd80c91e0 msgr2=0x7f5bd8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd80c91e0 0x7f5bd8069f60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009430 tx=0x7f5bd0017000 comp rx=0 tx=0).stop 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 shutdown_connections 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bbc060860 0x7f5bbc062d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bd8067ab0 0x7f5bd8068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd814f1e0 0x7f5bd8067570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 --2- 172.21.15.118:0/1764087424 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5bd80c91e0 0x7f5bd8069f60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 >> 172.21.15.118:0/1764087424 conn(0x7f5bd80c8620 msgr2=0x7f5bd80c19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:03.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 shutdown_connections 2024-08-18T13:40:03.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.094+0000 7f5bdea2b700 1 -- 172.21.15.118:0/1764087424 wait complete. 2024-08-18T13:40:03.108 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-18T13:40:03.109 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:40:03.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.202+0000 7f4aa1655700 1 Processor -- start 2024-08-18T13:40:03.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.202+0000 7f4aa1655700 1 -- start start 2024-08-18T13:40:03.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4a9c0add40 0x7f4a9c155b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4a9c0ae970 0x7f4a9c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a9c156060 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9c155110 con 0x7f4a9c0ae970 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9c06f340 con 0x7f4a9c0add40 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa0e54700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4a9c0ae970 0x7f4a9c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa0e54700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4a9c0ae970 0x7f4a9c15fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45478/0 (socket says 172.21.15.118:45478) 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa0e54700 1 -- 172.21.15.118:0/3546471239 learned_addr learned my addr 172.21.15.118:0/3546471239 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:03.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9bfff700 1 --2- 172.21.15.118:0/3546471239 >> v2:172.21.15.178:3300/0 conn(0x7f4a9c0add40 0x7f4a9c155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9bfff700 1 -- 172.21.15.118:0/3546471239 >> v2:172.21.15.118:3300/0 conn(0x7f4a9c0ae970 msgr2=0x7f4a9c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9bfff700 1 --2- 172.21.15.118:0/3546471239 >> v2:172.21.15.118:3300/0 conn(0x7f4a9c0ae970 0x7f4a9c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9bfff700 1 -- 172.21.15.118:0/3546471239 >> v1:172.21.15.178:6789/0 conn(0x7f4a9c0b15f0 legacy=0x7f4a9c0c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9bfff700 1 -- 172.21.15.118:0/3546471239 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0aeeb0 con 0x7f4a9c0add40 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa0e54700 1 --2- 172.21.15.118:0/3546471239 >> v2:172.21.15.118:3300/0 conn(0x7f4a9c0ae970 0x7f4a9c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9bfff700 1 --2- 172.21.15.118:0/3546471239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 0x7f4a9c155b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0717d0 tx=0x7f4a840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ad4c511e25af371 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9b7fe700 1 -- 172.21.15.118:0/3546471239 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84011630 con 0x7f4a9c0add40 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9b7fe700 1 -- 172.21.15.118:0/3546471239 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a84011790 con 0x7f4a9c0add40 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4a9b7fe700 1 -- 172.21.15.118:0/3546471239 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84011b30 con 0x7f4a9c0add40 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- 172.21.15.118:0/3546471239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 msgr2=0x7f4a9c155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 --2- 172.21.15.118:0/3546471239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 0x7f4a9c155b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0717d0 tx=0x7f4a840091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- 172.21.15.118:0/3546471239 shutdown_connections 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 --2- 172.21.15.118:0/3546471239 >> v2:172.21.15.118:3300/0 conn(0x7f4a9c0ae970 0x7f4a9c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 --2- 172.21.15.118:0/3546471239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 0x7f4a9c155b20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- 172.21.15.118:0/3546471239 >> 172.21.15.118:0/3546471239 conn(0x7f4a9c0cc600 msgr2=0x7f4a9c0c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:03.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- 172.21.15.118:0/3546471239 shutdown_connections 2024-08-18T13:40:03.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.206+0000 7f4aa1655700 1 -- 172.21.15.118:0/3546471239 wait complete. 2024-08-18T13:40:03.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 Processor -- start 2024-08-18T13:40:03.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 -- start start 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 0x7f4a9c165cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a9c0ae970 0x7f4a9c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a9c0b15f0 0x7f4a9c0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c165580 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c166a80 con 0x7f4a9c0add40 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c165820 con 0x7f4a9c0ae970 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a9c0b15f0 0x7f4a9c0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a9c0b15f0 0x7f4a9c0cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45492/0 (socket says 172.21.15.118:45492) 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 -- 172.21.15.118:0/4007062450 learned_addr learned my addr 172.21.15.118:0/4007062450 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 0x7f4a9c165cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4a937fe700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a9c0ae970 0x7f4a9c0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 -- 172.21.15.118:0/4007062450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a9c0ae970 msgr2=0x7f4a9c0cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a9c0ae970 0x7f4a9c0cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 -- 172.21.15.118:0/4007062450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 msgr2=0x7f4a9c165cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 0x7f4a9c165cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 -- 172.21.15.118:0/4007062450 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94009ae0 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa0e54700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a9c0b15f0 0x7f4a9c0cb1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400c0c0 tx=0x7f4a9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4a98ff9700 1 -- 172.21.15.118:0/4007062450 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94018070 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4a98ff9700 1 -- 172.21.15.118:0/4007062450 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9400f040 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a84009420 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4a98ff9700 1 -- 172.21.15.118:0/4007062450 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400ccb0 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c169880 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.210+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9c1659d0 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.214+0000 7f4a98ff9700 1 -- 172.21.15.118:0/4007062450 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4a94003020 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.214+0000 7f4a98ff9700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4a7c060860 0x7f4a7c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.214+0000 7f4a9bfff700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4a7c060860 0x7f4a7c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.214+0000 7f4a98ff9700 1 -- 172.21.15.118:0/4007062450 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f4a9407d250 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.214+0000 7f4a9bfff700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4a7c060860 0x7f4a7c062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4a840093f0 tx=0x7f4a84017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.226+0000 7f4a98ff9700 1 -- 172.21.15.118:0/4007062450 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4a9c1659d0 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.374+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a9c1659d0 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4a98ff9700 1 -- 172.21.15.118:0/4007062450 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f4a9c1659d0 con 0x7f4a9c0b15f0 2024-08-18T13:40:03.381 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:03.382 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4a7c060860 msgr2=0x7f4a7c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4a7c060860 0x7f4a7c062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4a840093f0 tx=0x7f4a84017000 comp rx=0 tx=0).stop 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a9c0b15f0 msgr2=0x7f4a9c0cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a9c0b15f0 0x7f4a9c0cb1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400c0c0 tx=0x7f4a9400b040 comp rx=0 tx=0).stop 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 shutdown_connections 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4a7c060860 0x7f4a7c062d10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4a9c0b15f0 0x7f4a9c0cb1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4a9c0ae970 0x7f4a9c0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 --2- 172.21.15.118:0/4007062450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9c0add40 0x7f4a9c165cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 >> 172.21.15.118:0/4007062450 conn(0x7f4a9c0cc600 msgr2=0x7f4a9c156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 shutdown_connections 2024-08-18T13:40:03.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.378+0000 7f4aa1655700 1 -- 172.21.15.118:0/4007062450 wait complete. 2024-08-18T13:40:03.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T13:39:21.246221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-18T13:39:32.826757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-18T13:39:33.731815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-18T13:39:37.447514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-18T13:40:03.394 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T13:40:03.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.502+0000 7fc79759e700 1 Processor -- start 2024-08-18T13:40:03.509 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.502+0000 7fc79759e700 1 -- start start 2024-08-18T13:40:03.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc7980c2f80 0x7fc7980c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980c3de0 con 0x7fc7980c3350 2024-08-18T13:40:03.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79806d620 con 0x7fc79814bff0 2024-08-18T13:40:03.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980bfb90 con 0x7fc7980c2f80 2024-08-18T13:40:03.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79659c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc7980c2f80 0x7fc7980c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc795d9b700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fc7980c3350 0x7fc79814b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:46016/0 (socket says 172.21.15.118:46016) 2024-08-18T13:40:03.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc795d9b700 1 -- 172.21.15.118:0/1401794061 learned_addr learned my addr 172.21.15.118:0/1401794061 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:03.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79659c700 1 -- 172.21.15.118:0/1401794061 >> v1:172.21.15.178:6789/0 conn(0x7fc79814bff0 legacy=0x7fc79815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79659c700 1 -- 172.21.15.118:0/1401794061 >> v1:172.21.15.118:6789/0 conn(0x7fc7980c3350 legacy=0x7fc79814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79659c700 1 -- 172.21.15.118:0/1401794061 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79806b820 con 0x7fc7980c2f80 2024-08-18T13:40:03.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79559a700 1 -- 172.21.15.118:0/1401794061 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310214782 0 0) 0x7fc7980c3de0 con 0x7fc7980c3350 2024-08-18T13:40:03.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79659c700 1 --2- 172.21.15.118:0/1401794061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c2f80 0x7fc7980c4270 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098e0 tx=0x7fc7880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2280f808f834f44b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79559a700 1 -- 172.21.15.118:0/1401794061 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788009e80 con 0x7fc7980c2f80 2024-08-18T13:40:03.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79559a700 1 -- 172.21.15.118:0/1401794061 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78800b860 con 0x7fc7980c2f80 2024-08-18T13:40:03.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79559a700 1 -- 172.21.15.118:0/1401794061 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800bbe0 con 0x7fc7980c2f80 2024-08-18T13:40:03.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 -- 172.21.15.118:0/1401794061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c2f80 msgr2=0x7fc7980c4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 --2- 172.21.15.118:0/1401794061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c2f80 0x7fc7980c4270 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098e0 tx=0x7fc7880091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 -- 172.21.15.118:0/1401794061 shutdown_connections 2024-08-18T13:40:03.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 --2- 172.21.15.118:0/1401794061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c2f80 0x7fc7980c4270 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 -- 172.21.15.118:0/1401794061 >> 172.21.15.118:0/1401794061 conn(0x7fc79806c750 msgr2=0x7fc7980c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:03.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.506+0000 7fc79759e700 1 -- 172.21.15.118:0/1401794061 shutdown_connections 2024-08-18T13:40:03.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 -- 172.21.15.118:0/1401794061 wait complete. 2024-08-18T13:40:03.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 Processor -- start 2024-08-18T13:40:03.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 -- start start 2024-08-18T13:40:03.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7980c2f80 0x7fc7981522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c3350 0x7fc79814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79814bff0 0x7fc79814cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798162a10 con 0x7fc7980c3350 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798161e20 con 0x7fc79814bff0 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798156090 con 0x7fc7980c2f80 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc795d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c3350 0x7fc79814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc795d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c3350 0x7fc79814c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45504/0 (socket says 172.21.15.118:45504) 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc795d9b700 1 -- 172.21.15.118:0/1720844193 learned_addr learned my addr 172.21.15.118:0/1720844193 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc79659c700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7980c2f80 0x7fc7981522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.510+0000 7fc796d9d700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79814bff0 0x7fc79814cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc795d9b700 1 -- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7980c2f80 msgr2=0x7fc7981522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc795d9b700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7980c2f80 0x7fc7981522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc795d9b700 1 -- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79814bff0 msgr2=0x7fc79814cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc795d9b700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79814bff0 0x7fc79814cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc795d9b700 1 -- 172.21.15.118:0/1720844193 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc788009420 con 0x7fc7980c3350 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc796d9d700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79814bff0 0x7fc79814cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc795d9b700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c3350 0x7fc79814c740 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00ea20 tx=0x7fc78c007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc7867fc700 1 -- 172.21.15.118:0/1720844193 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c001d60 con 0x7fc7980c3350 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc7867fc700 1 -- 172.21.15.118:0/1720844193 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c005b30 con 0x7fc7980c3350 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7981620e0 con 0x7fc7980c3350 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc7867fc700 1 -- 172.21.15.118:0/1720844193 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00c7e0 con 0x7fc7980c3350 2024-08-18T13:40:03.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7981626a0 con 0x7fc7980c3350 2024-08-18T13:40:03.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc798071410 con 0x7fc7980c3350 2024-08-18T13:40:03.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc7867fc700 1 -- 172.21.15.118:0/1720844193 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc78c00cb90 con 0x7fc7980c3350 2024-08-18T13:40:03.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc7867fc700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc778060860 0x7fc778062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc79659c700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc778060860 0x7fc778062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.514+0000 7fc7867fc700 1 -- 172.21.15.118:0/1720844193 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fc78c07f1d0 con 0x7fc7980c3350 2024-08-18T13:40:03.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.518+0000 7fc79659c700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc778060860 0x7fc778062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc7880093f0 tx=0x7fc788017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.526+0000 7fc7867fc700 1 -- 172.21.15.118:0/1720844193 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc78c04f3b0 con 0x7fc7980c3350 2024-08-18T13:40:03.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.690+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7980bf6c0 con 0x7fc7980c3350 2024-08-18T13:40:03.696 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.690+0000 7fc7867fc700 1 -- 172.21.15.118:0/1720844193 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc78c0529d0 con 0x7fc7980c3350 2024-08-18T13:40:03.696 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-08-18T13:40:03.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc778060860 msgr2=0x7fc778062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc778060860 0x7fc778062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc7880093f0 tx=0x7fc788017000 comp rx=0 tx=0).stop 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c3350 msgr2=0x7fc79814c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c3350 0x7fc79814c740 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc78c00ea20 tx=0x7fc78c007e20 comp rx=0 tx=0).stop 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 shutdown_connections 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc778060860 0x7fc778062d10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79814bff0 0x7fc79814cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7980c3350 0x7fc79814c740 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 --2- 172.21.15.118:0/1720844193 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7980c2f80 0x7fc7981522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 >> 172.21.15.118:0/1720844193 conn(0x7fc79806c750 msgr2=0x7fc798067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 shutdown_connections 2024-08-18T13:40:03.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.694+0000 7fc79759e700 1 -- 172.21.15.118:0/1720844193 wait complete. 2024-08-18T13:40:03.710 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-18T13:40:03.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064d3e8700 1 Processor -- start 2024-08-18T13:40:03.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064d3e8700 1 -- start start 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064d3e8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f064806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064d3e8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0648155ae0 0x7f0648155f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064d3e8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0648067a60 con 0x7f06480c13c0 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064d3e8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064806a770 con 0x7f0648155ae0 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064d3e8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06480c0f30 con 0x7f0648155710 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0648155ae0 0x7f0648155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0648155ae0 0x7f0648155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45518/0 (socket says 172.21.15.118:45518) 2024-08-18T13:40:03.821 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 -- 172.21.15.118:0/136538656 learned_addr learned my addr 172.21.15.118:0/136538656 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f064cbe7700 1 --1- 172.21.15.118:0/136538656 >> v1:172.21.15.118:6789/0 conn(0x7f06480c13c0 0x7f064814bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:46022/0 (socket says 172.21.15.118:46022) 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f0647fff700 1 --2- 172.21.15.118:0/136538656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f064806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 -- 172.21.15.118:0/136538656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 msgr2=0x7f064806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 --2- 172.21.15.118:0/136538656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f064806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 -- 172.21.15.118:0/136538656 >> v1:172.21.15.118:6789/0 conn(0x7f06480c13c0 legacy=0x7f064814bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f0646ffd700 1 -- 172.21.15.118:0/136538656 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688837708 0 0) 0x7f0648067a60 con 0x7f06480c13c0 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 -- 172.21.15.118:0/136538656 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648067780 con 0x7f0648155ae0 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f0647fff700 1 --2- 172.21.15.118:0/136538656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f064806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f06477fe700 1 --2- 172.21.15.118:0/136538656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648155f10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0630009f20 tx=0x7f0630009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2544bd8b72ddd65a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f0646ffd700 1 -- 172.21.15.118:0/136538656 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000e070 con 0x7f0648155ae0 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f0646ffd700 1 -- 172.21.15.118:0/136538656 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063000ba80 con 0x7f0648155ae0 2024-08-18T13:40:03.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.814+0000 7f0646ffd700 1 -- 172.21.15.118:0/136538656 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300092e0 con 0x7f0648155ae0 2024-08-18T13:40:03.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 -- 172.21.15.118:0/136538656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 msgr2=0x7f0648155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 --2- 172.21.15.118:0/136538656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648155f10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0630009f20 tx=0x7f0630009840 comp rx=0 tx=0).stop 2024-08-18T13:40:03.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 -- 172.21.15.118:0/136538656 shutdown_connections 2024-08-18T13:40:03.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 --2- 172.21.15.118:0/136538656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648155f10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 --2- 172.21.15.118:0/136538656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f064806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 -- 172.21.15.118:0/136538656 >> 172.21.15.118:0/136538656 conn(0x7f064806b1f0 msgr2=0x7f064806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:03.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 -- 172.21.15.118:0/136538656 shutdown_connections 2024-08-18T13:40:03.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.818+0000 7f064d3e8700 1 -- 172.21.15.118:0/136538656 wait complete. 2024-08-18T13:40:03.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 Processor -- start 2024-08-18T13:40:03.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 -- start start 2024-08-18T13:40:03.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06480c13c0 0x7f064806e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f0648161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648162010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480c3170 con 0x7f0648155ae0 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480c3cf0 con 0x7f06480c13c0 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06481437e0 con 0x7f0648155710 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648162010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648162010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45534/0 (socket says 172.21.15.118:45534) 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 -- 172.21.15.118:0/2869226323 learned_addr learned my addr 172.21.15.118:0/2869226323 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f06477fe700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f0648161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f0647fff700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06480c13c0 0x7f064806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 -- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 msgr2=0x7f0648161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f0648161ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 -- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06480c13c0 msgr2=0x7f064806e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06480c13c0 0x7f064806e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 -- 172.21.15.118:0/2869226323 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630009ae0 con 0x7f0648155ae0 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064cbe7700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648162010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f063800c610 tx=0x7f0638007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f063ffff700 1 -- 172.21.15.118:0/2869226323 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638005a70 con 0x7f0648155ae0 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f063ffff700 1 -- 172.21.15.118:0/2869226323 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0638005bd0 con 0x7f0648155ae0 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06480c33c0 con 0x7f0648155ae0 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f063ffff700 1 -- 172.21.15.118:0/2869226323 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380070d0 con 0x7f0648155ae0 2024-08-18T13:40:03.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.822+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06480c3980 con 0x7f0648155ae0 2024-08-18T13:40:03.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.826+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0648071410 con 0x7f0648155ae0 2024-08-18T13:40:03.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.826+0000 7f063ffff700 1 -- 172.21.15.118:0/2869226323 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0638007440 con 0x7f0648155ae0 2024-08-18T13:40:03.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.826+0000 7f063ffff700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f062c060860 0x7f062c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:03.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.826+0000 7f063ffff700 1 -- 172.21.15.118:0/2869226323 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f063807c350 con 0x7f0648155ae0 2024-08-18T13:40:03.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.826+0000 7f0647fff700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f062c060860 0x7f062c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:03.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.830+0000 7f0647fff700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f062c060860 0x7f062c062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0640006f00 tx=0x7f0640008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:03.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.834+0000 7f063ffff700 1 -- 172.21.15.118:0/2869226323 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f063804c4b0 con 0x7f0648155ae0 2024-08-18T13:40:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.998+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06480c3e30 con 0x7f0648155ae0 2024-08-18T13:40:04.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:03.998+0000 7f063ffff700 1 -- 172.21.15.118:0/2869226323 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06480c3e30 con 0x7f0648155ae0 2024-08-18T13:40:04.004 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:04.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f062c060860 msgr2=0x7f062c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f062c060860 0x7f062c062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0640006f00 tx=0x7f0640008040 comp rx=0 tx=0).stop 2024-08-18T13:40:04.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 msgr2=0x7f0648162010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648162010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f063800c610 tx=0x7f0638007d60 comp rx=0 tx=0).stop 2024-08-18T13:40:04.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 shutdown_connections 2024-08-18T13:40:04.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f062c060860 0x7f062c062d10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0648155ae0 0x7f0648162010 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0648155710 0x7f0648161ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 --2- 172.21.15.118:0/2869226323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06480c13c0 0x7f064806e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 >> 172.21.15.118:0/2869226323 conn(0x7f064806b1f0 msgr2=0x7f06480c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:04.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 shutdown_connections 2024-08-18T13:40:04.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.002+0000 7f064d3e8700 1 -- 172.21.15.118:0/2869226323 wait complete. 2024-08-18T13:40:04.018 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-18T13:40:04.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c43fff700 1 Processor -- start 2024-08-18T13:40:04.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c43fff700 1 -- start start 2024-08-18T13:40:04.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c43fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3c441552f0 0x7f3c440c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c43fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c44155750 0x7f3c440c3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c43fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c440c1ce0 con 0x7f3c44154f20 2024-08-18T13:40:04.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c43fff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c44068620 con 0x7f3c441552f0 2024-08-18T13:40:04.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c43fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4406a820 con 0x7f3c44155750 2024-08-18T13:40:04.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c42ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f3c44154f20 0x7f3c4406a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41070/0 (socket says 172.21.15.118:41070) 2024-08-18T13:40:04.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c42ffd700 1 -- 172.21.15.118:0/578999545 learned_addr learned my addr 172.21.15.118:0/578999545 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:04.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c437fe700 1 --2- 172.21.15.118:0/578999545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c44155750 0x7f3c440c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c427fc700 1 --2- 172.21.15.118:0/578999545 >> v2:172.21.15.178:3300/0 conn(0x7f3c441552f0 0x7f3c440c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.126+0000 7f3c41ffb700 1 -- 172.21.15.118:0/578999545 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779343210 0 0) 0x7f3c440c1ce0 con 0x7f3c44154f20 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c41ffb700 1 -- 172.21.15.118:0/578999545 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c30003570 con 0x7f3c44154f20 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c437fe700 1 -- 172.21.15.118:0/578999545 >> v2:172.21.15.178:3300/0 conn(0x7f3c441552f0 msgr2=0x7f3c440c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c437fe700 1 --2- 172.21.15.118:0/578999545 >> v2:172.21.15.178:3300/0 conn(0x7f3c441552f0 0x7f3c440c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c437fe700 1 -- 172.21.15.118:0/578999545 >> v1:172.21.15.118:6789/0 conn(0x7f3c44154f20 legacy=0x7f3c4406a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c437fe700 1 -- 172.21.15.118:0/578999545 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c44068810 con 0x7f3c44155750 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c427fc700 1 --2- 172.21.15.118:0/578999545 >> v2:172.21.15.178:3300/0 conn(0x7f3c441552f0 0x7f3c440c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c437fe700 1 --2- 172.21.15.118:0/578999545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c44155750 0x7f3c440c3360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3c340098e0 tx=0x7f3c340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1755c953a50bb78 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c41ffb700 1 -- 172.21.15.118:0/578999545 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34011630 con 0x7f3c44155750 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c41ffb700 1 -- 172.21.15.118:0/578999545 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c34011790 con 0x7f3c44155750 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c41ffb700 1 -- 172.21.15.118:0/578999545 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34011b10 con 0x7f3c44155750 2024-08-18T13:40:04.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- 172.21.15.118:0/578999545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c44155750 msgr2=0x7f3c440c3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 --2- 172.21.15.118:0/578999545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c44155750 0x7f3c440c3360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3c340098e0 tx=0x7f3c340091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- 172.21.15.118:0/578999545 shutdown_connections 2024-08-18T13:40:04.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 --2- 172.21.15.118:0/578999545 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c44155750 0x7f3c440c3360 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 --2- 172.21.15.118:0/578999545 >> v2:172.21.15.178:3300/0 conn(0x7f3c441552f0 0x7f3c440c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- 172.21.15.118:0/578999545 >> 172.21.15.118:0/578999545 conn(0x7f3c4406b1f0 msgr2=0x7f3c4406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:04.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- 172.21.15.118:0/578999545 shutdown_connections 2024-08-18T13:40:04.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- 172.21.15.118:0/578999545 wait complete. 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 Processor -- start 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- start start 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c44154f20 0x7f3c44151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c441552f0 0x7f3c4414d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c44155750 0x7f3c4414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4414a640 con 0x7f3c44154f20 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4414dba0 con 0x7f3c44155750 2024-08-18T13:40:04.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c43fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4414de80 con 0x7f3c441552f0 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c42ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c44154f20 0x7f3c44151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c42ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c44154f20 0x7f3c44151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58418/0 (socket says 172.21.15.118:58418) 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c42ffd700 1 -- 172.21.15.118:0/2040227020 learned_addr learned my addr 172.21.15.118:0/2040227020 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c427fc700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c441552f0 0x7f3c4414d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.130+0000 7f3c437fe700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c44155750 0x7f3c4414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c42ffd700 1 -- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c441552f0 msgr2=0x7f3c4414d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c42ffd700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c441552f0 0x7f3c4414d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c42ffd700 1 -- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c44155750 msgr2=0x7f3c4414f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c42ffd700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c44155750 0x7f3c4414f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c42ffd700 1 -- 172.21.15.118:0/2040227020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c009420 con 0x7f3c44154f20 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c437fe700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c44155750 0x7f3c4414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c42ffd700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c44154f20 0x7f3c44151d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00cc40 tx=0x7f3c3c001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c3affd700 1 -- 172.21.15.118:0/2040227020 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00a8d0 con 0x7f3c44154f20 2024-08-18T13:40:04.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c3affd700 1 -- 172.21.15.118:0/2040227020 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3c00aa30 con 0x7f3c44154f20 2024-08-18T13:40:04.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c34009420 con 0x7f3c44154f20 2024-08-18T13:40:04.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c3affd700 1 -- 172.21.15.118:0/2040227020 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00add0 con 0x7f3c44154f20 2024-08-18T13:40:04.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4414a2d0 con 0x7f3c44154f20 2024-08-18T13:40:04.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c3affd700 1 -- 172.21.15.118:0/2040227020 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3c3c007300 con 0x7f3c44154f20 2024-08-18T13:40:04.141 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.134+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c44071410 con 0x7f3c44154f20 2024-08-18T13:40:04.141 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.138+0000 7f3c3affd700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3c24060860 0x7f3c24062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.141 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.138+0000 7f3c3affd700 1 -- 172.21.15.118:0/2040227020 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f3c3c07d2c0 con 0x7f3c44154f20 2024-08-18T13:40:04.141 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.138+0000 7f3c427fc700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3c24060860 0x7f3c24062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.142 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.138+0000 7f3c427fc700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3c24060860 0x7f3c24062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3c440c1270 tx=0x7f3c2c0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.148 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.142+0000 7f3c3affd700 1 -- 172.21.15.118:0/2040227020 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3c3c04d380 con 0x7f3c44154f20 2024-08-18T13:40:04.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c4414f100 con 0x7f3c44154f20 2024-08-18T13:40:04.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c3affd700 1 -- 172.21.15.118:0/2040227020 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c3c0509a0 con 0x7f3c44154f20 2024-08-18T13:40:04.314 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 64 2024-08-18T13:40:04.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3c24060860 msgr2=0x7f3c24062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3c24060860 0x7f3c24062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3c440c1270 tx=0x7f3c2c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c44154f20 msgr2=0x7f3c44151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c44154f20 0x7f3c44151d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00cc40 tx=0x7f3c3c001a60 comp rx=0 tx=0).stop 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 shutdown_connections 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3c24060860 0x7f3c24062d10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c44155750 0x7f3c4414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c441552f0 0x7f3c4414d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 --2- 172.21.15.118:0/2040227020 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3c44154f20 0x7f3c44151d00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 >> 172.21.15.118:0/2040227020 conn(0x7f3c4406b1f0 msgr2=0x7f3c4406ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 shutdown_connections 2024-08-18T13:40:04.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.310+0000 7f3c43fff700 1 -- 172.21.15.118:0/2040227020 wait complete. 2024-08-18T13:40:04.328 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-18T13:40:04.444 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.438+0000 7faf13fff700 1 Processor -- start 2024-08-18T13:40:04.444 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.438+0000 7faf13fff700 1 -- start start 2024-08-18T13:40:04.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faf140c4220 0x7faf14069a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1414d4e0 0x7faf1415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf14153af0 con 0x7faf1406db00 2024-08-18T13:40:04.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf14152fd0 con 0x7faf140c4220 2024-08-18T13:40:04.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf14153f80 con 0x7faf1414d4e0 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faf140c4220 0x7faf14069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faf140c4220 0x7faf14069a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58430/0 (socket says 172.21.15.118:58430) 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 -- 172.21.15.118:0/2553558173 learned_addr learned my addr 172.21.15.118:0/2553558173 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf137fe700 1 --2- 172.21.15.118:0/2553558173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1414d4e0 0x7faf1415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 -- 172.21.15.118:0/2553558173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1414d4e0 msgr2=0x7faf1415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 --2- 172.21.15.118:0/2553558173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1414d4e0 0x7faf1415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 -- 172.21.15.118:0/2553558173 >> v1:172.21.15.178:6789/0 conn(0x7faf1406db00 legacy=0x7faf1414cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 -- 172.21.15.118:0/2553558173 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1405e850 con 0x7faf140c4220 2024-08-18T13:40:04.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf12ffd700 1 --2- 172.21.15.118:0/2553558173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf140c4220 0x7faf14069a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faf04009f50 tx=0x7faf04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d5a7fdd2bbf9f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf11ffb700 1 -- 172.21.15.118:0/2553558173 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400e070 con 0x7faf140c4220 2024-08-18T13:40:04.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf11ffb700 1 -- 172.21.15.118:0/2553558173 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0400b9c0 con 0x7faf140c4220 2024-08-18T13:40:04.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf11ffb700 1 -- 172.21.15.118:0/2553558173 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf040092e0 con 0x7faf140c4220 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- 172.21.15.118:0/2553558173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf140c4220 msgr2=0x7faf14069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 --2- 172.21.15.118:0/2553558173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf140c4220 0x7faf14069a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faf04009f50 tx=0x7faf04009840 comp rx=0 tx=0).stop 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- 172.21.15.118:0/2553558173 shutdown_connections 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 --2- 172.21.15.118:0/2553558173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1414d4e0 0x7faf1415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 --2- 172.21.15.118:0/2553558173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf140c4220 0x7faf14069a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- 172.21.15.118:0/2553558173 >> 172.21.15.118:0/2553558173 conn(0x7faf1406c750 msgr2=0x7faf140c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- 172.21.15.118:0/2553558173 shutdown_connections 2024-08-18T13:40:04.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.442+0000 7faf13fff700 1 -- 172.21.15.118:0/2553558173 wait complete. 2024-08-18T13:40:04.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 Processor -- start 2024-08-18T13:40:04.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 -- start start 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1406db00 0x7faf141632a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf140c4220 0x7faf14164720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf1414d4e0 0x7faf140c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1414bda0 con 0x7faf1414d4e0 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1414aeb0 con 0x7faf140c4220 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1414b190 con 0x7faf1406db00 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf1414d4e0 0x7faf140c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf1414d4e0 0x7faf140c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58444/0 (socket says 172.21.15.118:58444) 2024-08-18T13:40:04.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 -- 172.21.15.118:0/1019949276 learned_addr learned my addr 172.21.15.118:0/1019949276 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf12ffd700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1406db00 0x7faf141632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf127fc700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf140c4220 0x7faf14164720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 -- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1406db00 msgr2=0x7faf141632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1406db00 0x7faf141632a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 -- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf140c4220 msgr2=0x7faf14164720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf140c4220 0x7faf14164720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 -- 172.21.15.118:0/1019949276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04009ae0 con 0x7faf1414d4e0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf137fe700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf1414d4e0 0x7faf140c6ee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e8d0 tx=0x7faf0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf0affd700 1 -- 172.21.15.118:0/1019949276 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c013680 con 0x7faf1414d4e0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf0affd700 1 -- 172.21.15.118:0/1019949276 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c013800 con 0x7faf1414d4e0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.446+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1414b470 con 0x7faf1414d4e0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf0affd700 1 -- 172.21.15.118:0/1019949276 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c013bc0 con 0x7faf1414d4e0 2024-08-18T13:40:04.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1414ba30 con 0x7faf1414d4e0 2024-08-18T13:40:04.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1414bf50 con 0x7faf1414d4e0 2024-08-18T13:40:04.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf0affd700 1 -- 172.21.15.118:0/1019949276 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7faf0c0123d0 con 0x7faf1414d4e0 2024-08-18T13:40:04.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf0affd700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faef4060890 0x7faef4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf0affd700 1 -- 172.21.15.118:0/1019949276 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7faf0c07d290 con 0x7faf1414d4e0 2024-08-18T13:40:04.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf12ffd700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faef4060890 0x7faef4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.450+0000 7faf12ffd700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faef4060890 0x7faef4062d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c010 tx=0x7faf04018000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.458+0000 7faf0affd700 1 -- 172.21.15.118:0/1019949276 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faf1414bf50 con 0x7faf1414d4e0 2024-08-18T13:40:04.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.622+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faf14071410 con 0x7faf1414d4e0 2024-08-18T13:40:04.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.622+0000 7faf0affd700 1 -- 172.21.15.118:0/1019949276 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7faf0c04d3f0 con 0x7faf1414d4e0 2024-08-18T13:40:04.628 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:04.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faef4060890 msgr2=0x7faef4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faef4060890 0x7faef4062d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c010 tx=0x7faf04018000 comp rx=0 tx=0).stop 2024-08-18T13:40:04.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf1414d4e0 msgr2=0x7faf140c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf1414d4e0 0x7faf140c6ee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e8d0 tx=0x7faf0c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 shutdown_connections 2024-08-18T13:40:04.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faef4060890 0x7faef4062d40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf1414d4e0 0x7faf140c6ee0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf140c4220 0x7faf14164720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 --2- 172.21.15.118:0/1019949276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf1406db00 0x7faf141632a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 >> 172.21.15.118:0/1019949276 conn(0x7faf1406c750 msgr2=0x7faf14152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:04.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 shutdown_connections 2024-08-18T13:40:04.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.626+0000 7faf13fff700 1 -- 172.21.15.118:0/1019949276 wait complete. 2024-08-18T13:40:04.642 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:40:04.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.742+0000 7f2658e16700 1 Processor -- start 2024-08-18T13:40:04.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.742+0000 7f2658e16700 1 -- start start 2024-08-18T13:40:04.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2658e16700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f26540c4090 0x7f26540673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2658e16700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2654157360 con 0x7f2654067910 2024-08-18T13:40:04.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2658e16700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26540c3bd0 con 0x7f265414d6b0 2024-08-18T13:40:04.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2658e16700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26540c3eb0 con 0x7f26540c4090 2024-08-18T13:40:04.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2652d9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2654067910 0x7f265406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41078/0 (socket says 172.21.15.118:41078) 2024-08-18T13:40:04.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2652d9d700 1 -- 172.21.15.118:0/188690726 learned_addr learned my addr 172.21.15.118:0/188690726 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:04.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f264bfff700 1 --2- 172.21.15.118:0/188690726 >> v2:172.21.15.178:3300/0 conn(0x7f26540c4090 0x7f26540673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551724026 0 0) 0x7f2654157360 con 0x7f2654067910 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2634003570 con 0x7f2654067910 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 41778003 0 0) 0x7f2634003570 con 0x7f2654067910 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 >> v2:172.21.15.178:3300/0 conn(0x7f26540c4090 msgr2=0x7f26540673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 --2- 172.21.15.118:0/188690726 >> v2:172.21.15.178:3300/0 conn(0x7f26540c4090 0x7f26540673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 >> v1:172.21.15.178:6789/0 conn(0x7f265414d6b0 legacy=0x7f26540c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2654156a30 con 0x7f2654067910 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f264c0017f0 con 0x7f2654067910 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 >> v1:172.21.15.118:6789/0 conn(0x7f2654067910 legacy=0x7f265406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2634003a60 0x7f2634003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2634004810 0x7f2634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26340051c0 0x7f263400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540c3eb0 con 0x7f2634003a60 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263400c050 con 0x7f2634004810 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263400c210 con 0x7f26340051c0 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f265259c700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2634003a60 0x7f2634003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f264bfff700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2634004810 0x7f2634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2652d9d700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26340051c0 0x7f263400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f265259c700 1 -- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26340051c0 msgr2=0x7f263400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f265259c700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26340051c0 0x7f263400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f265259c700 1 -- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2634004810 msgr2=0x7f2634004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:04.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f265259c700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2634004810 0x7f2634004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f265259c700 1 -- 172.21.15.118:0/188690726 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634003570 con 0x7f2634003a60 2024-08-18T13:40:04.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2652d9d700 1 -- 172.21.15.118:0/188690726 reap_dead start 2024-08-18T13:40:04.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f265259c700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2634003a60 0x7f2634003e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26440019a0 tx=0x7f2644007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810dc84d5b7d9d2d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400c5d0 con 0x7f2634003a60 2024-08-18T13:40:04.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264400c730 con 0x7f2634003a60 2024-08-18T13:40:04.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.746+0000 7f2651d9b700 1 -- 172.21.15.118:0/188690726 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400cad0 con 0x7f2634003a60 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- 172.21.15.118:0/188690726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2634003a60 msgr2=0x7f2634003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2634003a60 0x7f2634003e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26440019a0 tx=0x7f2644007d10 comp rx=0 tx=0).stop 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- 172.21.15.118:0/188690726 shutdown_connections 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 --2- 172.21.15.118:0/188690726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2634003a60 0x7f2634003e30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- 172.21.15.118:0/188690726 >> 172.21.15.118:0/188690726 conn(0x7f26540c7c40 msgr2=0x7f26540c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- 172.21.15.118:0/188690726 shutdown_connections 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- 172.21.15.118:0/188690726 wait complete. 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 Processor -- start 2024-08-18T13:40:04.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- start start 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2634003a60 0x7f26541664d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2654166a10 0x7f265414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f265414faa0 0x7f26540bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265414f0d0 con 0x7f265414faa0 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265415ab20 con 0x7f2654166a10 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.750+0000 7f2658e16700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2654166e00 con 0x7f2634003a60 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f265414faa0 0x7f26540bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f264bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2654166a10 0x7f265414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f265414faa0 0x7f26540bfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58458/0 (socket says 172.21.15.118:58458) 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 -- 172.21.15.118:0/554891552 learned_addr learned my addr 172.21.15.118:0/554891552 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f265259c700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2634003a60 0x7f26541664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 -- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2634003a60 msgr2=0x7f26541664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2634003a60 0x7f26541664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 -- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2654166a10 msgr2=0x7f265414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2654166a10 0x7f265414f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 -- 172.21.15.118:0/554891552 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265414ff10 con 0x7f265414faa0 2024-08-18T13:40:04.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f264bfff700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2654166a10 0x7f265414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:04.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2652d9d700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f265414faa0 0x7f26540bfd70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f264c002250 tx=0x7f26541572d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f264a7fc700 1 -- 172.21.15.118:0/554891552 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c00aa90 con 0x7f265414faa0 2024-08-18T13:40:04.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f264a7fc700 1 -- 172.21.15.118:0/554891552 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c00abf0 con 0x7f265414faa0 2024-08-18T13:40:04.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f264a7fc700 1 -- 172.21.15.118:0/554891552 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c0133f0 con 0x7f265414faa0 2024-08-18T13:40:04.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2654167070 con 0x7f265414faa0 2024-08-18T13:40:04.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.754+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265414ed60 con 0x7f265414faa0 2024-08-18T13:40:04.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.758+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26540642f0 con 0x7f265414faa0 2024-08-18T13:40:04.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.758+0000 7f264a7fc700 1 -- 172.21.15.118:0/554891552 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f264c013820 con 0x7f265414faa0 2024-08-18T13:40:04.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.758+0000 7f264a7fc700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2638060860 0x7f2638062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:04.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.758+0000 7f264a7fc700 1 -- 172.21.15.118:0/554891552 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f264c07d2a0 con 0x7f265414faa0 2024-08-18T13:40:04.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.758+0000 7f265259c700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2638060860 0x7f2638062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:04.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.758+0000 7f265259c700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2638060860 0x7f2638062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2644000c00 tx=0x7f2644015000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:04.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.766+0000 7f264a7fc700 1 -- 172.21.15.118:0/554891552 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f264c04d400 con 0x7f265414faa0 2024-08-18T13:40:04.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.962+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26540c2f70 con 0x7f265414faa0 2024-08-18T13:40:04.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.966+0000 7f264a7fc700 1 -- 172.21.15.118:0/554891552 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7f264c050a20 con 0x7f265414faa0 2024-08-18T13:40:04.970 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:04.970 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2638060860 msgr2=0x7f2638062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2638060860 0x7f2638062d10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2644000c00 tx=0x7f2644015000 comp rx=0 tx=0).stop 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f265414faa0 msgr2=0x7f26540bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f265414faa0 0x7f26540bfd70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f264c002250 tx=0x7f26541572d0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 shutdown_connections 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2638060860 0x7f2638062d10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f265414faa0 0x7f26540bfd70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2654166a10 0x7f265414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 --2- 172.21.15.118:0/554891552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2634003a60 0x7f26541664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:04.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 >> 172.21.15.118:0/554891552 conn(0x7f26540c7c40 msgr2=0x7f265406b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:04.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 shutdown_connections 2024-08-18T13:40:04.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:04.970+0000 7f2658e16700 1 -- 172.21.15.118:0/554891552 wait complete. 2024-08-18T13:40:04.975 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:40:04.990 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:40:05.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.094+0000 7f5d0302e700 1 Processor -- start 2024-08-18T13:40:05.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.094+0000 7f5d0302e700 1 -- start start 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5cfc0c8200 0x7f5cfc0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5cfc06f2f0 0x7f5cfc0c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc06ee60 con 0x7f5cfc153690 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc15a640 con 0x7f5cfc0c8200 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc06f730 con 0x7f5cfc06f2f0 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5cfc0c8200 0x7f5cfc0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5cfc0c8200 0x7f5cfc0673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58462/0 (socket says 172.21.15.118:58462) 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 -- 172.21.15.118:0/3392212162 learned_addr learned my addr 172.21.15.118:0/3392212162 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:05.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0282d700 1 --2- 172.21.15.118:0/3392212162 >> v2:172.21.15.178:3300/0 conn(0x7f5cfc06f2f0 0x7f5cfc0c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 -- 172.21.15.118:0/3392212162 >> v2:172.21.15.178:3300/0 conn(0x7f5cfc06f2f0 msgr2=0x7f5cfc0c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:05.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 --2- 172.21.15.118:0/3392212162 >> v2:172.21.15.178:3300/0 conn(0x7f5cfc06f2f0 0x7f5cfc0c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 -- 172.21.15.118:0/3392212162 >> v1:172.21.15.178:6789/0 conn(0x7f5cfc153690 legacy=0x7f5cfc0c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 -- 172.21.15.118:0/3392212162 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc153d10 con 0x7f5cfc0c8200 2024-08-18T13:40:05.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0182b700 1 --2- 172.21.15.118:0/3392212162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc0c8200 0x7f5cfc0673d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc06aa00 tx=0x7f5cf4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e92e1bf401faf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0102a700 1 -- 172.21.15.118:0/3392212162 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400e070 con 0x7f5cfc0c8200 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0102a700 1 -- 172.21.15.118:0/3392212162 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400ba20 con 0x7f5cfc0c8200 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0102a700 1 -- 172.21.15.118:0/3392212162 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40092e0 con 0x7f5cfc0c8200 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- 172.21.15.118:0/3392212162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc0c8200 msgr2=0x7f5cfc0673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 --2- 172.21.15.118:0/3392212162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc0c8200 0x7f5cfc0673d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc06aa00 tx=0x7f5cf4009840 comp rx=0 tx=0).stop 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- 172.21.15.118:0/3392212162 shutdown_connections 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 --2- 172.21.15.118:0/3392212162 >> v2:172.21.15.178:3300/0 conn(0x7f5cfc06f2f0 0x7f5cfc0c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 --2- 172.21.15.118:0/3392212162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc0c8200 0x7f5cfc0673d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- 172.21.15.118:0/3392212162 >> 172.21.15.118:0/3392212162 conn(0x7f5cfc06c020 msgr2=0x7f5cfc0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:05.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- 172.21.15.118:0/3392212162 shutdown_connections 2024-08-18T13:40:05.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.098+0000 7f5d0302e700 1 -- 172.21.15.118:0/3392212162 wait complete. 2024-08-18T13:40:05.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 Processor -- start 2024-08-18T13:40:05.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 -- start start 2024-08-18T13:40:05.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cfc06f2f0 0x7f5cfc1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cfc0c8200 0x7f5cfc169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc153690 0x7f5cfc16a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c6af0 con 0x7f5cfc153690 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc169950 con 0x7f5cfc06f2f0 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0302e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c6810 con 0x7f5cfc0c8200 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0282d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc153690 0x7f5cfc16a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0202c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cfc06f2f0 0x7f5cfc1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0282d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc153690 0x7f5cfc16a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58466/0 (socket says 172.21.15.118:58466) 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0182b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cfc0c8200 0x7f5cfc169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0282d700 1 -- 172.21.15.118:0/1239264876 learned_addr learned my addr 172.21.15.118:0/1239264876 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0282d700 1 -- 172.21.15.118:0/1239264876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cfc0c8200 msgr2=0x7f5cfc169da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.102+0000 7f5d0282d700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cfc0c8200 0x7f5cfc169da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0282d700 1 -- 172.21.15.118:0/1239264876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cfc06f2f0 msgr2=0x7f5cfc1562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0282d700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cfc06f2f0 0x7f5cfc1562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0282d700 1 -- 172.21.15.118:0/1239264876 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4009ae0 con 0x7f5cfc153690 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0202c700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cfc06f2f0 0x7f5cfc1562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0282d700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc153690 0x7f5cfc16a2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800e8d0 tx=0x7f5cf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5cf27fc700 1 -- 172.21.15.118:0/1239264876 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8013680 con 0x7f5cfc153690 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5cf27fc700 1 -- 172.21.15.118:0/1239264876 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf80137e0 con 0x7f5cfc153690 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc0b4050 con 0x7f5cfc153690 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5cf27fc700 1 -- 172.21.15.118:0/1239264876 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8013ba0 con 0x7f5cfc153690 2024-08-18T13:40:05.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc0b5110 con 0x7f5cfc153690 2024-08-18T13:40:05.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc169ac0 con 0x7f5cfc153690 2024-08-18T13:40:05.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.106+0000 7f5cf27fc700 1 -- 172.21.15.118:0/1239264876 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5cf80123d0 con 0x7f5cfc153690 2024-08-18T13:40:05.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.110+0000 7f5cf27fc700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5ce8060860 0x7f5ce8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.110+0000 7f5cf27fc700 1 -- 172.21.15.118:0/1239264876 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f5cf807d2a0 con 0x7f5cfc153690 2024-08-18T13:40:05.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.110+0000 7f5d0202c700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5ce8060860 0x7f5ce8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.110+0000 7f5d0202c700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5ce8060860 0x7f5ce8062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cec001dc0 tx=0x7f5cec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.120 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.114+0000 7f5cf27fc700 1 -- 172.21.15.118:0/1239264876 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5cfc169ac0 con 0x7f5cfc153690 2024-08-18T13:40:05.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.274+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cfc169ac0 con 0x7f5cfc153690 2024-08-18T13:40:05.282 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5cf27fc700 1 -- 172.21.15.118:0/1239264876 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f5cfc169ac0 con 0x7f5cfc153690 2024-08-18T13:40:05.283 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:05.283 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:40:05.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5ce8060860 msgr2=0x7f5ce8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5ce8060860 0x7f5ce8062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cec001dc0 tx=0x7f5cec009040 comp rx=0 tx=0).stop 2024-08-18T13:40:05.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc153690 msgr2=0x7f5cfc16a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc153690 0x7f5cfc16a2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800e8d0 tx=0x7f5cf80091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 shutdown_connections 2024-08-18T13:40:05.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5ce8060860 0x7f5ce8062d10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cfc153690 0x7f5cfc16a2e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.278+0000 7f5d0302e700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5cfc0c8200 0x7f5cfc169da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.282+0000 7f5d0302e700 1 --2- 172.21.15.118:0/1239264876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5cfc06f2f0 0x7f5cfc1562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.282+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 >> 172.21.15.118:0/1239264876 conn(0x7f5cfc06c020 msgr2=0x7f5cfc152cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:05.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.282+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 shutdown_connections 2024-08-18T13:40:05.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.282+0000 7f5d0302e700 1 -- 172.21.15.118:0/1239264876 wait complete. 2024-08-18T13:40:05.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T13:39:21.246221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-18T13:39:32.826757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-18T13:39:33.731815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-18T13:39:37.447514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-18T13:40:05.301 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T13:40:05.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e32c6700 1 Processor -- start 2024-08-18T13:40:05.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e32c6700 1 -- start start 2024-08-18T13:40:05.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e32c6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96dc0c78b0 0x7f96dc158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e32c6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc157480 0x7f96dc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e32c6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc0ac1b0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e32c6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc0ac3e0 con 0x7f96dc0c78b0 2024-08-18T13:40:05.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e32c6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0ac680 con 0x7f96dc157480 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc157480 0x7f96dc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1062700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96dc0c78b0 0x7f96dc158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc157480 0x7f96dc0c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:46350/0 (socket says 172.21.15.118:46350) 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 -- 172.21.15.118:0/1397657867 learned_addr learned my addr 172.21.15.118:0/1397657867 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1062700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96dc0c78b0 0x7f96dc158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:36228/0 (socket says 172.21.15.118:36228) 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 -- 172.21.15.118:0/1397657867 >> v2:172.21.15.178:3300/0 conn(0x7f96dc0c78b0 msgr2=0x7f96dc158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 --2- 172.21.15.118:0/1397657867 >> v2:172.21.15.178:3300/0 conn(0x7f96dc0c78b0 0x7f96dc158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 -- 172.21.15.118:0/1397657867 >> v1:172.21.15.178:6789/0 conn(0x7f96dc0c7c80 legacy=0x7f96dc156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 -- 172.21.15.118:0/1397657867 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc0ac960 con 0x7f96dc157480 2024-08-18T13:40:05.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.414+0000 7f96e1863700 1 --2- 172.21.15.118:0/1397657867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc157480 0x7f96dc0c8080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009f20 tx=0x7f96d4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7af3c3fa47ea20c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96d3fff700 1 -- 172.21.15.118:0/1397657867 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400e070 con 0x7f96dc157480 2024-08-18T13:40:05.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96d3fff700 1 -- 172.21.15.118:0/1397657867 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d400ba60 con 0x7f96dc157480 2024-08-18T13:40:05.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96d3fff700 1 -- 172.21.15.118:0/1397657867 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d40092e0 con 0x7f96dc157480 2024-08-18T13:40:05.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- 172.21.15.118:0/1397657867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc157480 msgr2=0x7f96dc0c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 --2- 172.21.15.118:0/1397657867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc157480 0x7f96dc0c8080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009f20 tx=0x7f96d4009840 comp rx=0 tx=0).stop 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- 172.21.15.118:0/1397657867 shutdown_connections 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 --2- 172.21.15.118:0/1397657867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc157480 0x7f96dc0c8080 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 --2- 172.21.15.118:0/1397657867 >> v2:172.21.15.178:3300/0 conn(0x7f96dc0c78b0 0x7f96dc158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- 172.21.15.118:0/1397657867 >> 172.21.15.118:0/1397657867 conn(0x7f96dc0cc810 msgr2=0x7f96dc0c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- 172.21.15.118:0/1397657867 shutdown_connections 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- 172.21.15.118:0/1397657867 wait complete. 2024-08-18T13:40:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 Processor -- start 2024-08-18T13:40:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- start start 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96dc0c78b0 0x7f96dc067c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc0c7c80 0x7f96dc168f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96dc157480 0x7f96dc1694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0affe0 con 0x7f96dc0c78b0 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc152a40 con 0x7f96dc157480 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.418+0000 7f96e32c6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0af5b0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc0c7c80 0x7f96dc168f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e1863700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96dc157480 0x7f96dc1694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc0c7c80 0x7f96dc168f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:46360/0 (socket says 172.21.15.118:46360) 2024-08-18T13:40:05.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 -- 172.21.15.118:0/2865781268 learned_addr learned my addr 172.21.15.118:0/2865781268 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:05.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e1062700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96dc0c78b0 0x7f96dc067c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 -- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96dc157480 msgr2=0x7f96dc1694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96dc157480 0x7f96dc1694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 -- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96dc0c78b0 msgr2=0x7f96dc067c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96dc0c78b0 0x7f96dc067c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 -- 172.21.15.118:0/2865781268 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009ae0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e1863700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96dc157480 0x7f96dc1694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e0861700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc0c7c80 0x7f96dc168f60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c790 tx=0x7f96d8007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96d17fa700 1 -- 172.21.15.118:0/2865781268 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8011590 con 0x7f96dc0c7c80 2024-08-18T13:40:05.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96d17fa700 1 -- 172.21.15.118:0/2865781268 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d8011710 con 0x7f96dc0c7c80 2024-08-18T13:40:05.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96d17fa700 1 -- 172.21.15.118:0/2865781268 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8011ad0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc152780 con 0x7f96dc0c7c80 2024-08-18T13:40:05.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc0aef60 con 0x7f96dc0c7c80 2024-08-18T13:40:05.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0af720 con 0x7f96dc0c7c80 2024-08-18T13:40:05.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.422+0000 7f96d17fa700 1 -- 172.21.15.118:0/2865781268 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f96d801b3b0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.426+0000 7f96d17fa700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96c0060890 0x7f96c0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.426+0000 7f96d17fa700 1 -- 172.21.15.118:0/2865781268 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f96d807c250 con 0x7f96dc0c7c80 2024-08-18T13:40:05.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.426+0000 7f96e1062700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96c0060890 0x7f96c0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.426+0000 7f96e1062700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96c0060890 0x7f96c0062d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009db0 tx=0x7f96cc0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.434+0000 7f96d17fa700 1 -- 172.21.15.118:0/2865781268 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f96d804c3b0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.598+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96dc0648f0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.602+0000 7f96d17fa700 1 -- 172.21.15.118:0/2865781268 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96d804f9d0 con 0x7f96dc0c7c80 2024-08-18T13:40:05.605 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-08-18T13:40:05.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96c0060890 msgr2=0x7f96c0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96c0060890 0x7f96c0062d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009db0 tx=0x7f96cc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc0c7c80 msgr2=0x7f96dc168f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc0c7c80 0x7f96dc168f60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c790 tx=0x7f96d8007f40 comp rx=0 tx=0).stop 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 shutdown_connections 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f96c0060890 0x7f96c0062d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96dc157480 0x7f96dc1694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96dc0c7c80 0x7f96dc168f60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 --2- 172.21.15.118:0/2865781268 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96dc0c78b0 0x7f96dc067c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 >> 172.21.15.118:0/2865781268 conn(0x7f96dc0cc810 msgr2=0x7f96dc0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 shutdown_connections 2024-08-18T13:40:05.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.606+0000 7f96e32c6700 1 -- 172.21.15.118:0/2865781268 wait complete. 2024-08-18T13:40:05.624 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-18T13:40:05.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.726+0000 7fbbef442700 1 Processor -- start 2024-08-18T13:40:05.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.726+0000 7fbbef442700 1 -- start start 2024-08-18T13:40:05.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbef442700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbbe80c9570 0x7fbbe81560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbef442700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe80c9ba0 con 0x7fbbe806edb0 2024-08-18T13:40:05.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbef442700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe80673d0 con 0x7fbbe80bfc10 2024-08-18T13:40:05.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbef442700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe80676b0 con 0x7fbbe80c9570 2024-08-18T13:40:05.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbed1de700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbbe806edb0 0x7fbbe80688a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41082/0 (socket says 172.21.15.118:41082) 2024-08-18T13:40:05.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbed1de700 1 -- 172.21.15.118:0/2826912673 learned_addr learned my addr 172.21.15.118:0/2826912673 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:05.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbec9dd700 1 --2- 172.21.15.118:0/2826912673 >> v2:172.21.15.178:3300/0 conn(0x7fbbe80c9570 0x7fbbe81560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbdffff700 1 -- 172.21.15.118:0/2826912673 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843479454 0 0) 0x7fbbe80c9ba0 con 0x7fbbe806edb0 2024-08-18T13:40:05.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbdffff700 1 -- 172.21.15.118:0/2826912673 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd4003570 con 0x7fbbe806edb0 2024-08-18T13:40:05.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbdffff700 1 -- 172.21.15.118:0/2826912673 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4114995637 0 0) 0x7fbbd4003570 con 0x7fbbe806edb0 2024-08-18T13:40:05.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbec9dd700 1 -- 172.21.15.118:0/2826912673 >> v1:172.21.15.178:6789/0 conn(0x7fbbe80bfc10 legacy=0x7fbbe8152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbec9dd700 1 -- 172.21.15.118:0/2826912673 >> v1:172.21.15.118:6789/0 conn(0x7fbbe806edb0 legacy=0x7fbbe80688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbec9dd700 1 -- 172.21.15.118:0/2826912673 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8067990 con 0x7fbbe80c9570 2024-08-18T13:40:05.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbec9dd700 1 --2- 172.21.15.118:0/2826912673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81560a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009910 tx=0x7fbbe40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c316eccc9fd07137 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbdffff700 1 -- 172.21.15.118:0/2826912673 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8001470 con 0x7fbbe80c9570 2024-08-18T13:40:05.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbdffff700 1 -- 172.21.15.118:0/2826912673 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4011420 con 0x7fbbe80c9570 2024-08-18T13:40:05.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbdffff700 1 -- 172.21.15.118:0/2826912673 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40117c0 con 0x7fbbe80c9570 2024-08-18T13:40:05.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbef442700 1 -- 172.21.15.118:0/2826912673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 msgr2=0x7fbbe81560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.730+0000 7fbbef442700 1 --2- 172.21.15.118:0/2826912673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81560a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009910 tx=0x7fbbe40091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.734+0000 7fbbef442700 1 -- 172.21.15.118:0/2826912673 shutdown_connections 2024-08-18T13:40:05.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.734+0000 7fbbef442700 1 --2- 172.21.15.118:0/2826912673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81560a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.734+0000 7fbbef442700 1 -- 172.21.15.118:0/2826912673 >> 172.21.15.118:0/2826912673 conn(0x7fbbe806d1b0 msgr2=0x7fbbe80c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:05.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.734+0000 7fbbef442700 1 -- 172.21.15.118:0/2826912673 shutdown_connections 2024-08-18T13:40:05.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.734+0000 7fbbef442700 1 -- 172.21.15.118:0/2826912673 wait complete. 2024-08-18T13:40:05.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.734+0000 7fbbef442700 1 Processor -- start 2024-08-18T13:40:05.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.734+0000 7fbbef442700 1 -- start start 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe806edb0 0x7fbbe8152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe80bfc10 0x7fbbe8154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe815b030 con 0x7fbbe80bfc10 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80ba500 con 0x7fbbe80c9570 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe81661a0 con 0x7fbbe806edb0 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe80bfc10 0x7fbbe8154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe80bfc10 0x7fbbe8154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58488/0 (socket says 172.21.15.118:58488) 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 -- 172.21.15.118:0/3887268296 learned_addr learned my addr 172.21.15.118:0/3887268296 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:05.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbed1de700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe806edb0 0x7fbbe8152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbed9df700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 -- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe806edb0 msgr2=0x7fbbe8152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe806edb0 0x7fbbe8152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 -- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 msgr2=0x7fbbe81551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 -- 172.21.15.118:0/3887268296 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe4009420 con 0x7fbbe80bfc10 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbed9df700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbec9dd700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe80bfc10 0x7fbbe8154cb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4003080 tx=0x7fbbe4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbdd7fa700 1 -- 172.21.15.118:0/3887268296 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401ba10 con 0x7fbbe80bfc10 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbdd7fa700 1 -- 172.21.15.118:0/3887268296 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4020070 con 0x7fbbe80bfc10 2024-08-18T13:40:05.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe815b580 con 0x7fbbe80bfc10 2024-08-18T13:40:05.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbdd7fa700 1 -- 172.21.15.118:0/3887268296 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401bda0 con 0x7fbbe80bfc10 2024-08-18T13:40:05.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe815b820 con 0x7fbbe80bfc10 2024-08-18T13:40:05.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe815adf0 con 0x7fbbe80bfc10 2024-08-18T13:40:05.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.738+0000 7fbbdd7fa700 1 -- 172.21.15.118:0/3887268296 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbbe4022030 con 0x7fbbe80bfc10 2024-08-18T13:40:05.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.742+0000 7fbbdd7fa700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbcc060860 0x7fbbcc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:05.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.742+0000 7fbbdd7fa700 1 -- 172.21.15.118:0/3887268296 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fbbe407e240 con 0x7fbbe80bfc10 2024-08-18T13:40:05.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.742+0000 7fbbed1de700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbcc060860 0x7fbbcc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:05.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.742+0000 7fbbed1de700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbcc060860 0x7fbbcc062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8001ed0 tx=0x7fbbd8000bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:05.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.746+0000 7fbbdd7fa700 1 -- 172.21.15.118:0/3887268296 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbbe815adf0 con 0x7fbbe80bfc10 2024-08-18T13:40:05.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.910+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbe815adf0 con 0x7fbbe80bfc10 2024-08-18T13:40:05.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.910+0000 7fbbdd7fa700 1 -- 172.21.15.118:0/3887268296 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbe815adf0 con 0x7fbbe80bfc10 2024-08-18T13:40:05.917 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:05.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbcc060860 msgr2=0x7fbbcc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbcc060860 0x7fbbcc062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8001ed0 tx=0x7fbbd8000bc0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe80bfc10 msgr2=0x7fbbe8154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:05.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe80bfc10 0x7fbbe8154cb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4003080 tx=0x7fbbe4016000 comp rx=0 tx=0).stop 2024-08-18T13:40:05.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 shutdown_connections 2024-08-18T13:40:05.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbcc060860 0x7fbbcc062d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbe80c9570 0x7fbbe81551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbe80bfc10 0x7fbbe8154cb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 --2- 172.21.15.118:0/3887268296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbe806edb0 0x7fbbe8152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:05.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 >> 172.21.15.118:0/3887268296 conn(0x7fbbe806d1b0 msgr2=0x7fbbe80688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:05.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 shutdown_connections 2024-08-18T13:40:05.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:05.914+0000 7fbbef442700 1 -- 172.21.15.118:0/3887268296 wait complete. 2024-08-18T13:40:05.931 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-18T13:40:06.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.038+0000 7f659ffff700 1 Processor -- start 2024-08-18T13:40:06.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.038+0000 7f659ffff700 1 -- start start 2024-08-18T13:40:06.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a014f3b0 0x7f65a0155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a00bf700 con 0x7f65a00ca760 2024-08-18T13:40:06.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a01595e0 con 0x7f65a014f780 2024-08-18T13:40:06.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a01585d0 con 0x7f65a014f3b0 2024-08-18T13:40:06.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659e7fc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f65a00ca760 0x7f65a01587c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41090/0 (socket says 172.21.15.118:41090) 2024-08-18T13:40:06.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659e7fc700 1 -- 172.21.15.118:0/3484592173 learned_addr learned my addr 172.21.15.118:0/3484592173 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:06.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659effd700 1 --2- 172.21.15.118:0/3484592173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a014f3b0 0x7f65a0155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659dffb700 1 -- 172.21.15.118:0/3484592173 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674109378 0 0) 0x7f65a00bf700 con 0x7f65a00ca760 2024-08-18T13:40:06.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659dffb700 1 -- 172.21.15.118:0/3484592173 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6588003570 con 0x7f65a00ca760 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659dffb700 1 -- 172.21.15.118:0/3484592173 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876655174 0 0) 0x7f65a01595e0 con 0x7f65a014f780 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659dffb700 1 -- 172.21.15.118:0/3484592173 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a00bf700 con 0x7f65a014f780 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659effd700 1 -- 172.21.15.118:0/3484592173 >> v1:172.21.15.178:6789/0 conn(0x7f65a014f780 legacy=0x7f65a0158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659effd700 1 -- 172.21.15.118:0/3484592173 >> v1:172.21.15.118:6789/0 conn(0x7f65a00ca760 legacy=0x7f65a01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659effd700 1 -- 172.21.15.118:0/3484592173 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00aeb10 con 0x7f65a014f3b0 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659effd700 1 --2- 172.21.15.118:0/3484592173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a014f3b0 0x7f65a0155cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6590009910 tx=0x7f65900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c817431dc1589a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659dffb700 1 -- 172.21.15.118:0/3484592173 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000b690 con 0x7f65a014f3b0 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659dffb700 1 -- 172.21.15.118:0/3484592173 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659000b7f0 con 0x7f65a014f3b0 2024-08-18T13:40:06.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659dffb700 1 -- 172.21.15.118:0/3484592173 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000bb90 con 0x7f65a014f3b0 2024-08-18T13:40:06.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- 172.21.15.118:0/3484592173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a014f3b0 msgr2=0x7f65a0155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 --2- 172.21.15.118:0/3484592173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a014f3b0 0x7f65a0155cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6590009910 tx=0x7f65900091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- 172.21.15.118:0/3484592173 shutdown_connections 2024-08-18T13:40:06.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 --2- 172.21.15.118:0/3484592173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a014f3b0 0x7f65a0155cc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- 172.21.15.118:0/3484592173 >> 172.21.15.118:0/3484592173 conn(0x7f65a006e090 msgr2=0x7f65a00c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:06.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- 172.21.15.118:0/3484592173 shutdown_connections 2024-08-18T13:40:06.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.042+0000 7f659ffff700 1 -- 172.21.15.118:0/3484592173 wait complete. 2024-08-18T13:40:06.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 Processor -- start 2024-08-18T13:40:06.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 -- start start 2024-08-18T13:40:06.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a00ca760 0x7f65a00c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a014f3b0 0x7f65a00ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65a014f780 0x7f65a0151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0152920 con 0x7f65a014f780 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0169190 con 0x7f65a014f3b0 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00abbd0 con 0x7f65a00ca760 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a014f3b0 0x7f65a00ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65a014f780 0x7f65a0151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659effd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a00ca760 0x7f65a00c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a014f3b0 0x7f65a00ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:36274/0 (socket says 172.21.15.118:36274) 2024-08-18T13:40:06.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659e7fc700 1 -- 172.21.15.118:0/1139652604 learned_addr learned my addr 172.21.15.118:0/1139652604 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659f7fe700 1 -- 172.21.15.118:0/1139652604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a00ca760 msgr2=0x7f65a00c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659f7fe700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a00ca760 0x7f65a00c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659f7fe700 1 -- 172.21.15.118:0/1139652604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a014f3b0 msgr2=0x7f65a00ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659f7fe700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a014f3b0 0x7f65a00ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659f7fe700 1 -- 172.21.15.118:0/1139652604 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009420 con 0x7f65a014f780 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659f7fe700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65a014f780 0x7f65a0151000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f65a006f770 tx=0x7f6598002650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f6586ffd700 1 -- 172.21.15.118:0/1139652604 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598012630 con 0x7f65a014f780 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f6586ffd700 1 -- 172.21.15.118:0/1139652604 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65980127b0 con 0x7f65a014f780 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.046+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a00c73d0 con 0x7f65a014f780 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f6586ffd700 1 -- 172.21.15.118:0/1139652604 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598012b70 con 0x7f65a014f780 2024-08-18T13:40:06.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a0168860 con 0x7f65a014f780 2024-08-18T13:40:06.055 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a0169340 con 0x7f65a014f780 2024-08-18T13:40:06.055 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f6586ffd700 1 -- 172.21.15.118:0/1139652604 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f659801b400 con 0x7f65a014f780 2024-08-18T13:40:06.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f6586ffd700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f658c060860 0x7f658c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f6586ffd700 1 -- 172.21.15.118:0/1139652604 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f659807d0d0 con 0x7f65a014f780 2024-08-18T13:40:06.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f659effd700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f658c060860 0x7f658c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.050+0000 7f659effd700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f658c060860 0x7f658c062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6590009860 tx=0x7f6590017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.058+0000 7f6586ffd700 1 -- 172.21.15.118:0/1139652604 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f65a0169340 con 0x7f65a014f780 2024-08-18T13:40:06.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.226+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65a0169340 con 0x7f65a014f780 2024-08-18T13:40:06.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.226+0000 7f6586ffd700 1 -- 172.21.15.118:0/1139652604 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65a0169340 con 0x7f65a014f780 2024-08-18T13:40:06.231 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 64 2024-08-18T13:40:06.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f658c060860 msgr2=0x7f658c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f658c060860 0x7f658c062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6590009860 tx=0x7f6590017000 comp rx=0 tx=0).stop 2024-08-18T13:40:06.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65a014f780 msgr2=0x7f65a0151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65a014f780 0x7f65a0151000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f65a006f770 tx=0x7f6598002650 comp rx=0 tx=0).stop 2024-08-18T13:40:06.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 shutdown_connections 2024-08-18T13:40:06.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f658c060860 0x7f658c062d10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f65a014f780 0x7f65a0151000 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a014f3b0 0x7f65a00ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 --2- 172.21.15.118:0/1139652604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65a00ca760 0x7f65a00c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 >> 172.21.15.118:0/1139652604 conn(0x7f65a006e090 msgr2=0x7f65a0154080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:06.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 shutdown_connections 2024-08-18T13:40:06.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.230+0000 7f659ffff700 1 -- 172.21.15.118:0/1139652604 wait complete. 2024-08-18T13:40:06.246 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-18T13:40:06.357 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.350+0000 7ff7f8fd3700 1 Processor -- start 2024-08-18T13:40:06.357 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f8fd3700 1 -- start start 2024-08-18T13:40:06.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f8fd3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff7f40c5b60 0x7ff7f4156930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f8fd3700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff7f4069b30 0x7ff7f4156e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f8fd3700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f40673d0 con 0x7ff7f4069760 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f8fd3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f4069f20 con 0x7ff7f40c5b60 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f8fd3700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40c2760 con 0x7ff7f4069b30 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f2ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff7f40c5b60 0x7ff7f4156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f37fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff7f4069760 0x7ff7f40c2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41102/0 (socket says 172.21.15.118:41102) 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f37fe700 1 -- 172.21.15.118:0/3404632165 learned_addr learned my addr 172.21.15.118:0/3404632165 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f2ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff7f40c5b60 0x7ff7f4156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58496/0 (socket says 172.21.15.118:58496) 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f3fff700 1 --2- 172.21.15.118:0/3404632165 >> v2:172.21.15.178:3300/0 conn(0x7ff7f4069b30 0x7ff7f4156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f27fc700 1 -- 172.21.15.118:0/3404632165 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196202933 0 0) 0x7ff7f40673d0 con 0x7ff7f4069760 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f27fc700 1 -- 172.21.15.118:0/3404632165 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d4003570 con 0x7ff7f4069760 2024-08-18T13:40:06.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f2ffd700 1 -- 172.21.15.118:0/3404632165 >> v2:172.21.15.178:3300/0 conn(0x7ff7f4069b30 msgr2=0x7ff7f4156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:06.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f2ffd700 1 --2- 172.21.15.118:0/3404632165 >> v2:172.21.15.178:3300/0 conn(0x7ff7f4069b30 0x7ff7f4156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f2ffd700 1 -- 172.21.15.118:0/3404632165 >> v1:172.21.15.118:6789/0 conn(0x7ff7f4069760 legacy=0x7ff7f40c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f2ffd700 1 -- 172.21.15.118:0/3404632165 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4067f50 con 0x7ff7f40c5b60 2024-08-18T13:40:06.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f2ffd700 1 --2- 172.21.15.118:0/3404632165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f40c5b60 0x7ff7f4156930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40c5330 tx=0x7ff7e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a243e1ee243ecc6e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f27fc700 1 -- 172.21.15.118:0/3404632165 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400e070 con 0x7ff7f40c5b60 2024-08-18T13:40:06.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f27fc700 1 -- 172.21.15.118:0/3404632165 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400bab0 con 0x7ff7f40c5b60 2024-08-18T13:40:06.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.354+0000 7ff7f27fc700 1 -- 172.21.15.118:0/3404632165 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400be70 con 0x7ff7f40c5b60 2024-08-18T13:40:06.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3404632165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f40c5b60 msgr2=0x7ff7f4156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3404632165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f40c5b60 0x7ff7f4156930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40c5330 tx=0x7ff7e4009780 comp rx=0 tx=0).stop 2024-08-18T13:40:06.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3404632165 shutdown_connections 2024-08-18T13:40:06.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3404632165 >> v2:172.21.15.178:3300/0 conn(0x7ff7f4069b30 0x7ff7f4156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3404632165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f40c5b60 0x7ff7f4156930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3404632165 >> 172.21.15.118:0/3404632165 conn(0x7ff7f40c9400 msgr2=0x7ff7f40c5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:06.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3404632165 shutdown_connections 2024-08-18T13:40:06.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3404632165 wait complete. 2024-08-18T13:40:06.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 Processor -- start 2024-08-18T13:40:06.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- start start 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f4069760 0x7ff7f40c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7f4069b30 0x7ff7f4142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7f40c5b60 0x7ff7f41434c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4165eb0 con 0x7ff7f4069760 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4166750 con 0x7ff7f40c5b60 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f8fd3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4156100 con 0x7ff7f4069b30 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f4069760 0x7ff7f40c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f4069760 0x7ff7f40c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58512/0 (socket says 172.21.15.118:58512) 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 -- 172.21.15.118:0/3080233105 learned_addr learned my addr 172.21.15.118:0/3080233105 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f2ffd700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7f4069b30 0x7ff7f4142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f3fff700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7f40c5b60 0x7ff7f41434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 -- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7f4069b30 msgr2=0x7ff7f4142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7f4069b30 0x7ff7f4142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 -- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7f40c5b60 msgr2=0x7ff7f41434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7f40c5b60 0x7ff7f41434c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 -- 172.21.15.118:0/3080233105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e4009a20 con 0x7ff7f4069760 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.358+0000 7ff7f37fe700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f4069760 0x7ff7f40c3040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c7f0 tx=0x7ff7dc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7eb7fe700 1 -- 172.21.15.118:0/3080233105 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc005cd0 con 0x7ff7f4069760 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7eb7fe700 1 -- 172.21.15.118:0/3080233105 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc005e50 con 0x7ff7f4069760 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f41563e0 con 0x7ff7f4069760 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7eb7fe700 1 -- 172.21.15.118:0/3080233105 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc007320 con 0x7ff7f4069760 2024-08-18T13:40:06.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f41663e0 con 0x7ff7f4069760 2024-08-18T13:40:06.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f4071410 con 0x7ff7f4069760 2024-08-18T13:40:06.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7eb7fe700 1 -- 172.21.15.118:0/3080233105 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff7dc01b3b0 con 0x7ff7f4069760 2024-08-18T13:40:06.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7eb7fe700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff7d8060860 0x7ff7d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7eb7fe700 1 -- 172.21.15.118:0/3080233105 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7ff7dc07c5b0 con 0x7ff7f4069760 2024-08-18T13:40:06.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7f2ffd700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff7d8060860 0x7ff7d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.362+0000 7ff7f2ffd700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff7d8060860 0x7ff7d8062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097c0 tx=0x7ff7e4019000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.370+0000 7ff7eb7fe700 1 -- 172.21.15.118:0/3080233105 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff7dc04c670 con 0x7ff7f4069760 2024-08-18T13:40:06.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.534+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff7f405e6b0 con 0x7ff7f4069760 2024-08-18T13:40:06.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.534+0000 7ff7eb7fe700 1 -- 172.21.15.118:0/3080233105 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff7dc04fc90 con 0x7ff7f4069760 2024-08-18T13:40:06.540 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:06.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff7d8060860 msgr2=0x7ff7d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff7d8060860 0x7ff7d8062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40097c0 tx=0x7ff7e4019000 comp rx=0 tx=0).stop 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f4069760 msgr2=0x7ff7f40c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f4069760 0x7ff7f40c3040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c7f0 tx=0x7ff7dc007e00 comp rx=0 tx=0).stop 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 shutdown_connections 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff7d8060860 0x7ff7d8062d10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7f40c5b60 0x7ff7f41434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7f4069b30 0x7ff7f4142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 --2- 172.21.15.118:0/3080233105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7f4069760 0x7ff7f40c3040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 >> 172.21.15.118:0/3080233105 conn(0x7ff7f40c9400 msgr2=0x7ff7f414b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 shutdown_connections 2024-08-18T13:40:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.538+0000 7ff7f8fd3700 1 -- 172.21.15.118:0/3080233105 wait complete. 2024-08-18T13:40:06.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:40:06.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 Processor -- start 2024-08-18T13:40:06.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- start start 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f965806c6a0 0x7f9658155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f965806ca70 0x7f96580c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96580c1e80 con 0x7f96580c90e0 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96581563d0 con 0x7f965806c6a0 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96580c9690 con 0x7f965806ca70 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f965806c6a0 0x7f9658155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f965806c6a0 0x7f9658155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58522/0 (socket says 172.21.15.118:58522) 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 -- 172.21.15.118:0/3329455191 learned_addr learned my addr 172.21.15.118:0/3329455191 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:06.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965ce41700 1 --2- 172.21.15.118:0/3329455191 >> v2:172.21.15.178:3300/0 conn(0x7f965806ca70 0x7f96580c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 -- 172.21.15.118:0/3329455191 >> v2:172.21.15.178:3300/0 conn(0x7f965806ca70 msgr2=0x7f96580c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 --2- 172.21.15.118:0/3329455191 >> v2:172.21.15.178:3300/0 conn(0x7f965806ca70 0x7f96580c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 -- 172.21.15.118:0/3329455191 >> v1:172.21.15.178:6789/0 conn(0x7f96580c90e0 legacy=0x7f965814bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 -- 172.21.15.118:0/3329455191 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580c1550 con 0x7f965806c6a0 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965d642700 1 --2- 172.21.15.118:0/3329455191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806c6a0 0x7f9658155e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f965806d8e0 tx=0x7f9648009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae288f95196c60b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f964ffff700 1 -- 172.21.15.118:0/3329455191 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648009e70 con 0x7f965806c6a0 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f964ffff700 1 -- 172.21.15.118:0/3329455191 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964800b8f0 con 0x7f965806c6a0 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f964ffff700 1 -- 172.21.15.118:0/3329455191 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800bc90 con 0x7f965806c6a0 2024-08-18T13:40:06.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- 172.21.15.118:0/3329455191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806c6a0 msgr2=0x7f9658155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 --2- 172.21.15.118:0/3329455191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806c6a0 0x7f9658155e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f965806d8e0 tx=0x7f9648009250 comp rx=0 tx=0).stop 2024-08-18T13:40:06.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- 172.21.15.118:0/3329455191 shutdown_connections 2024-08-18T13:40:06.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 --2- 172.21.15.118:0/3329455191 >> v2:172.21.15.178:3300/0 conn(0x7f965806ca70 0x7f96580c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 --2- 172.21.15.118:0/3329455191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806c6a0 0x7f9658155e90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- 172.21.15.118:0/3329455191 >> 172.21.15.118:0/3329455191 conn(0x7f965806f4d0 msgr2=0x7f96580c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:06.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- 172.21.15.118:0/3329455191 shutdown_connections 2024-08-18T13:40:06.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.654+0000 7f965e644700 1 -- 172.21.15.118:0/3329455191 wait complete. 2024-08-18T13:40:06.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.658+0000 7f965e644700 1 Processor -- start 2024-08-18T13:40:06.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.658+0000 7f965e644700 1 -- start start 2024-08-18T13:40:06.666 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965806c6a0 0x7f96580c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806ca70 0x7f965814cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96580c90e0 0x7f96580c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658163db0 con 0x7f965806ca70 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580c4bd0 con 0x7f96580c90e0 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9658163b10 con 0x7f965806c6a0 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806ca70 0x7f965814cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806ca70 0x7f965814cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58524/0 (socket says 172.21.15.118:58524) 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 -- 172.21.15.118:0/1657016885 learned_addr learned my addr 172.21.15.118:0/1657016885 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965d642700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965806c6a0 0x7f96580c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965de43700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96580c90e0 0x7f96580c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.667 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 -- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965806c6a0 msgr2=0x7f96580c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965806c6a0 0x7f96580c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 -- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96580c90e0 msgr2=0x7f96580c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96580c90e0 0x7f96580c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 -- 172.21.15.118:0/1657016885 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96480094f0 con 0x7f965806ca70 2024-08-18T13:40:06.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965ce41700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806ca70 0x7f965814cbe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9650009ef0 tx=0x7f965000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f964d7fa700 1 -- 172.21.15.118:0/1657016885 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965000cb70 con 0x7f965806ca70 2024-08-18T13:40:06.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f964d7fa700 1 -- 172.21.15.118:0/1657016885 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965000f040 con 0x7f965806ca70 2024-08-18T13:40:06.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965814f590 con 0x7f965806ca70 2024-08-18T13:40:06.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f964d7fa700 1 -- 172.21.15.118:0/1657016885 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650014480 con 0x7f965806ca70 2024-08-18T13:40:06.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.662+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96580c4580 con 0x7f965806ca70 2024-08-18T13:40:06.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.666+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96580c8840 con 0x7f965806ca70 2024-08-18T13:40:06.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.666+0000 7f964d7fa700 1 -- 172.21.15.118:0/1657016885 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f96500147f0 con 0x7f965806ca70 2024-08-18T13:40:06.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.666+0000 7f964d7fa700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f963c060890 0x7f963c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:06.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.666+0000 7f964d7fa700 1 -- 172.21.15.118:0/1657016885 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f965007da30 con 0x7f965806ca70 2024-08-18T13:40:06.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.666+0000 7f965d642700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f963c060890 0x7f963c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:06.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.666+0000 7f965d642700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f963c060890 0x7f963c062d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f965806d8e0 tx=0x7f9648017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:06.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.674+0000 7f964d7fa700 1 -- 172.21.15.118:0/1657016885 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f96580c8840 con 0x7f965806ca70 2024-08-18T13:40:06.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.874+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96580c8840 con 0x7f965806ca70 2024-08-18T13:40:06.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.874+0000 7f964d7fa700 1 -- 172.21.15.118:0/1657016885 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7f96580c8840 con 0x7f965806ca70 2024-08-18T13:40:06.880 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:06.880 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f963c060890 msgr2=0x7f963c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f963c060890 0x7f963c062d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f965806d8e0 tx=0x7f9648017000 comp rx=0 tx=0).stop 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806ca70 msgr2=0x7f965814cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806ca70 0x7f965814cbe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9650009ef0 tx=0x7f965000b040 comp rx=0 tx=0).stop 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 shutdown_connections 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f963c060890 0x7f963c062d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96580c90e0 0x7f96580c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f965806ca70 0x7f965814cbe0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 --2- 172.21.15.118:0/1657016885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f965806c6a0 0x7f96580c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:06.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 >> 172.21.15.118:0/1657016885 conn(0x7f965806f4d0 msgr2=0x7f9658148b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:06.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 shutdown_connections 2024-08-18T13:40:06.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.878+0000 7f965e644700 1 -- 172.21.15.118:0/1657016885 wait complete. 2024-08-18T13:40:06.883 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:40:06.894 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:40:07.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.994+0000 7f9ffe104700 1 Processor -- start 2024-08-18T13:40:07.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.994+0000 7f9ffe104700 1 -- start start 2024-08-18T13:40:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.994+0000 7f9ffe104700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9ff814f330 0x7f9ff80adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.994+0000 7f9ffe104700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff815fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.994+0000 7f9ffe104700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff80bfe20 con 0x7f9ff8158720 2024-08-18T13:40:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff80ad3f0 con 0x7f9ff814f330 2024-08-18T13:40:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80bef50 con 0x7f9ff80c02b0 2024-08-18T13:40:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffc901700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f9ff8158720 0x7f9ff806bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41118/0 (socket says 172.21.15.118:41118) 2024-08-18T13:40:07.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffc901700 1 -- 172.21.15.118:0/2135744943 learned_addr learned my addr 172.21.15.118:0/2135744943 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:07.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffd903700 1 --2- 172.21.15.118:0/2135744943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff815fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffd102700 1 --2- 172.21.15.118:0/2135744943 >> v2:172.21.15.178:3300/0 conn(0x7f9ff814f330 0x7f9ff80adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ff7fff700 1 -- 172.21.15.118:0/2135744943 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931217674 0 0) 0x7f9ff80bfe20 con 0x7f9ff8158720 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ff7fff700 1 -- 172.21.15.118:0/2135744943 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fe0003570 con 0x7f9ff8158720 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffd903700 1 -- 172.21.15.118:0/2135744943 >> v2:172.21.15.178:3300/0 conn(0x7f9ff814f330 msgr2=0x7f9ff80adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffd903700 1 --2- 172.21.15.118:0/2135744943 >> v2:172.21.15.178:3300/0 conn(0x7f9ff814f330 0x7f9ff80adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffd903700 1 -- 172.21.15.118:0/2135744943 >> v1:172.21.15.118:6789/0 conn(0x7f9ff8158720 legacy=0x7f9ff806bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffd903700 1 -- 172.21.15.118:0/2135744943 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff80bf210 con 0x7f9ff80c02b0 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffd903700 1 --2- 172.21.15.118:0/2135744943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff815fac0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00098e0 tx=0x7f9ff00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65badcb486f5da45 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ff7fff700 1 -- 172.21.15.118:0/2135744943 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0011630 con 0x7f9ff80c02b0 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ff7fff700 1 -- 172.21.15.118:0/2135744943 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff0011790 con 0x7f9ff80c02b0 2024-08-18T13:40:07.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ff7fff700 1 -- 172.21.15.118:0/2135744943 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0011b30 con 0x7f9ff80c02b0 2024-08-18T13:40:07.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 -- 172.21.15.118:0/2135744943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 msgr2=0x7f9ff815fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 --2- 172.21.15.118:0/2135744943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff815fac0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00098e0 tx=0x7f9ff00091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 -- 172.21.15.118:0/2135744943 shutdown_connections 2024-08-18T13:40:07.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 --2- 172.21.15.118:0/2135744943 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff815fac0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 --2- 172.21.15.118:0/2135744943 >> v2:172.21.15.178:3300/0 conn(0x7f9ff814f330 0x7f9ff80adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 -- 172.21.15.118:0/2135744943 >> 172.21.15.118:0/2135744943 conn(0x7f9ff80ca6d0 msgr2=0x7f9ff806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:07.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 -- 172.21.15.118:0/2135744943 shutdown_connections 2024-08-18T13:40:07.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:06.998+0000 7f9ffe104700 1 -- 172.21.15.118:0/2135744943 wait complete. 2024-08-18T13:40:07.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 Processor -- start 2024-08-18T13:40:07.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 -- start start 2024-08-18T13:40:07.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff8154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ff814f330 0x7f9ff8155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8158720 0x7f9ff81538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff8166280 con 0x7f9ff814f330 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80ac0d0 con 0x7f9ff8158720 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff8165fa0 con 0x7f9ff80c02b0 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ff814f330 0x7f9ff8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ff814f330 0x7f9ff8155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58532/0 (socket says 172.21.15.118:58532) 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 -- 172.21.15.118:0/533211480 learned_addr learned my addr 172.21.15.118:0/533211480 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffd102700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff8154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffd903700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8158720 0x7f9ff81538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 -- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 msgr2=0x7f9ff8154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff8154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 -- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8158720 msgr2=0x7f9ff81538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8158720 0x7f9ff81538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 -- 172.21.15.118:0/533211480 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009420 con 0x7f9ff814f330 2024-08-18T13:40:07.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffd102700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff8154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:07.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffc901700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ff814f330 0x7f9ff8155420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00cc40 tx=0x7f9fec001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:07.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ff57fa700 1 -- 172.21.15.118:0/533211480 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00a910 con 0x7f9ff814f330 2024-08-18T13:40:07.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ff57fa700 1 -- 172.21.15.118:0/533211480 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec00aa90 con 0x7f9ff814f330 2024-08-18T13:40:07.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ff57fa700 1 -- 172.21.15.118:0/533211480 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00ae50 con 0x7f9ff814f330 2024-08-18T13:40:07.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff81533b0 con 0x7f9ff814f330 2024-08-18T13:40:07.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.002+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff8150190 con 0x7f9ff814f330 2024-08-18T13:40:07.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.006+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff8165e00 con 0x7f9ff814f330 2024-08-18T13:40:07.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.006+0000 7f9ff57fa700 1 -- 172.21.15.118:0/533211480 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9fec007300 con 0x7f9ff814f330 2024-08-18T13:40:07.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.006+0000 7f9ff57fa700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9fe4060890 0x7f9fe4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.006+0000 7f9ff57fa700 1 -- 172.21.15.118:0/533211480 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f9fec07d2a0 con 0x7f9ff814f330 2024-08-18T13:40:07.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.006+0000 7f9ffd102700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9fe4060890 0x7f9fe4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.006+0000 7f9ffd102700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9fe4060890 0x7f9fe4062d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009d80 tx=0x7f9fe8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:07.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.010+0000 7f9ff57fa700 1 -- 172.21.15.118:0/533211480 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9ff8165e00 con 0x7f9ff814f330 2024-08-18T13:40:07.182 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.178+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ff8155ae0 con 0x7f9ff814f330 2024-08-18T13:40:07.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.178+0000 7f9ff57fa700 1 -- 172.21.15.118:0/533211480 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f9ff8155ae0 con 0x7f9ff814f330 2024-08-18T13:40:07.185 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:07.185 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:40:07.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9fe4060890 msgr2=0x7f9fe4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9fe4060890 0x7f9fe4062d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009d80 tx=0x7f9fe8008040 comp rx=0 tx=0).stop 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ff814f330 msgr2=0x7f9ff8155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ff814f330 0x7f9ff8155420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00cc40 tx=0x7f9fec001a70 comp rx=0 tx=0).stop 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 shutdown_connections 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9fe4060890 0x7f9fe4062d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8158720 0x7f9ff81538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9ff814f330 0x7f9ff8155420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 --2- 172.21.15.118:0/533211480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ff80c02b0 0x7f9ff8154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 >> 172.21.15.118:0/533211480 conn(0x7f9ff80ca6d0 msgr2=0x7f9ff814e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 shutdown_connections 2024-08-18T13:40:07.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.182+0000 7f9ffe104700 1 -- 172.21.15.118:0/533211480 wait complete. 2024-08-18T13:40:07.199 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:40:07.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.298+0000 7f34a759e700 1 Processor -- start 2024-08-18T13:40:07.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.298+0000 7f34a759e700 1 -- start start 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f34a814b940 0x7f34a806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f34a80be5a0 0x7f34a806ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a8151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a806fac0 con 0x7f34a814b940 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a8151870 con 0x7f34a80be5a0 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a806de00 con 0x7f34a814bd10 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f34a814b940 0x7f34a806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f34a814b940 0x7f34a806e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58540/0 (socket says 172.21.15.118:58540) 2024-08-18T13:40:07.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f349ffff700 1 --2- 172.21.15.118:0/48494506 >> v2:172.21.15.178:3300/0 conn(0x7f34a80be5a0 0x7f34a806ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 -- 172.21.15.118:0/48494506 learned_addr learned my addr 172.21.15.118:0/48494506 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:07.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a6d9d700 1 --2- 172.21.15.118:0/48494506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a8151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 -- 172.21.15.118:0/48494506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 msgr2=0x7f34a8151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 --2- 172.21.15.118:0/48494506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a8151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 -- 172.21.15.118:0/48494506 >> v2:172.21.15.178:3300/0 conn(0x7f34a80be5a0 msgr2=0x7f34a806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 --2- 172.21.15.118:0/48494506 >> v2:172.21.15.178:3300/0 conn(0x7f34a80be5a0 0x7f34a806ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 -- 172.21.15.118:0/48494506 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a80c2da0 con 0x7f34a814b940 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f349ffff700 1 --2- 172.21.15.118:0/48494506 >> v2:172.21.15.178:3300/0 conn(0x7f34a80be5a0 0x7f34a806ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a659c700 1 --2- 172.21.15.118:0/48494506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a806e4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f34a806d470 tx=0x7f3498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95929f5f442728b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:07.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a5d9b700 1 -- 172.21.15.118:0/48494506 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349800b640 con 0x7f34a814b940 2024-08-18T13:40:07.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a5d9b700 1 -- 172.21.15.118:0/48494506 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349800b7a0 con 0x7f34a814b940 2024-08-18T13:40:07.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a5d9b700 1 -- 172.21.15.118:0/48494506 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349800bb60 con 0x7f34a814b940 2024-08-18T13:40:07.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- 172.21.15.118:0/48494506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 msgr2=0x7f34a806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 --2- 172.21.15.118:0/48494506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a806e4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f34a806d470 tx=0x7f3498009250 comp rx=0 tx=0).stop 2024-08-18T13:40:07.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- 172.21.15.118:0/48494506 shutdown_connections 2024-08-18T13:40:07.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 --2- 172.21.15.118:0/48494506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 --2- 172.21.15.118:0/48494506 >> v2:172.21.15.178:3300/0 conn(0x7f34a80be5a0 0x7f34a806ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 --2- 172.21.15.118:0/48494506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a806e4c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- 172.21.15.118:0/48494506 >> 172.21.15.118:0/48494506 conn(0x7f34a806c020 msgr2=0x7f34a80c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:07.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- 172.21.15.118:0/48494506 shutdown_connections 2024-08-18T13:40:07.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.302+0000 7f34a759e700 1 -- 172.21.15.118:0/48494506 wait complete. 2024-08-18T13:40:07.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 Processor -- start 2024-08-18T13:40:07.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 -- start start 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a80be5a0 0x7f34a81638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a8146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a81471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8164f10 con 0x7f34a814b940 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8140c50 con 0x7f34a80be5a0 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8140ef0 con 0x7f34a814bd10 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a8146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a8146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58554/0 (socket says 172.21.15.118:58554) 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 -- 172.21.15.118:0/2309003737 learned_addr learned my addr 172.21.15.118:0/2309003737 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:07.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a6d9d700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a81471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a659c700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a80be5a0 0x7f34a81638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 -- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 msgr2=0x7f34a81471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a81471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 -- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a80be5a0 msgr2=0x7f34a81638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a80be5a0 0x7f34a81638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 -- 172.21.15.118:0/2309003737 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490009420 con 0x7f34a814b940 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349ffff700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a8146c60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f349000ec80 tx=0x7f34900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349e7fc700 1 -- 172.21.15.118:0/2309003737 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000c8e0 con 0x7f34a814b940 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349e7fc700 1 -- 172.21.15.118:0/2309003737 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000ca40 con 0x7f34a814b940 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34980094f0 con 0x7f34a814b940 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f349e7fc700 1 -- 172.21.15.118:0/2309003737 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000cde0 con 0x7f34a814b940 2024-08-18T13:40:07.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.306+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a81414b0 con 0x7f34a814b940 2024-08-18T13:40:07.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.310+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a80713a0 con 0x7f34a814b940 2024-08-18T13:40:07.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.310+0000 7f349e7fc700 1 -- 172.21.15.118:0/2309003737 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3490013680 con 0x7f34a814b940 2024-08-18T13:40:07.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.310+0000 7f349e7fc700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3488060890 0x7f3488062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:07.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.310+0000 7f349e7fc700 1 -- 172.21.15.118:0/2309003737 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f349007ce50 con 0x7f34a814b940 2024-08-18T13:40:07.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.310+0000 7f34a659c700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3488060890 0x7f3488062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:07.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.310+0000 7f34a659c700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3488060890 0x7f3488062d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34a806d470 tx=0x7f3498017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:07.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.318+0000 7f349e7fc700 1 -- 172.21.15.118:0/2309003737 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f349004cfb0 con 0x7f34a814b940 2024-08-18T13:40:07.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.522+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34a80683e0 con 0x7f34a814b940 2024-08-18T13:40:07.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.522+0000 7f349e7fc700 1 -- 172.21.15.118:0/2309003737 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7f3490017090 con 0x7f34a814b940 2024-08-18T13:40:07.529 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:07.529 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:40:07.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3488060890 msgr2=0x7f3488062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3488060890 0x7f3488062d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f34a806d470 tx=0x7f3498017000 comp rx=0 tx=0).stop 2024-08-18T13:40:07.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 msgr2=0x7f34a8146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:07.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a8146c60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f349000ec80 tx=0x7f34900091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 shutdown_connections 2024-08-18T13:40:07.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3488060890 0x7f3488062d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34a814bd10 0x7f34a81471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f34a814b940 0x7f34a8146c60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 --2- 172.21.15.118:0/2309003737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a80be5a0 0x7f34a81638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:07.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 >> 172.21.15.118:0/2309003737 conn(0x7f34a806c020 msgr2=0x7f34a8144980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:07.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 shutdown_connections 2024-08-18T13:40:07.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:07.526+0000 7f34a759e700 1 -- 172.21.15.118:0/2309003737 wait complete. 2024-08-18T13:40:07.533 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:40:08.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-18T13:39:34.735047+0000', 'modified': '2024-08-18T13:39:57.488947+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4312': {'gid': 4312, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.178:6837/2503344632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 2503344632}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 2503344632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-18T13:40:08.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-18T13:40:08.147 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-08-18T13:40:08.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71ce0aa700 1 Processor -- start 2024-08-18T13:40:08.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71ce0aa700 1 -- start start 2024-08-18T13:40:08.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71ce0aa700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f71c81556f0 0x7f71c80c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71ce0aa700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8151f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71ce0aa700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80c1110 con 0x7f71c80684e0 2024-08-18T13:40:08.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71ce0aa700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80c0b90 con 0x7f71c81556f0 2024-08-18T13:40:08.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71ce0aa700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80c8060 con 0x7f71c81561d0 2024-08-18T13:40:08.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.262+0000 7f71cd0a8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f71c81556f0 0x7f71c80c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd0a8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f71c81556f0 0x7f71c80c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58562/0 (socket says 172.21.15.118:58562) 2024-08-18T13:40:08.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd0a8700 1 -- 172.21.15.118:0/308886625 learned_addr learned my addr 172.21.15.118:0/308886625 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/308886625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd0a8700 1 -- 172.21.15.118:0/308886625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 msgr2=0x7f71c8151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71bffff700 1 -- 172.21.15.118:0/308886625 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097533685 0 0) 0x7f71c80c1110 con 0x7f71c80684e0 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/308886625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd0a8700 1 -- 172.21.15.118:0/308886625 >> v1:172.21.15.178:6789/0 conn(0x7f71c80684e0 legacy=0x7f71c8155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd0a8700 1 -- 172.21.15.118:0/308886625 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80c8300 con 0x7f71c81556f0 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/308886625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/308886625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c81556f0 0x7f71c80c15a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71c40099b0 tx=0x7f71c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d00d6e430372615 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71bffff700 1 -- 172.21.15.118:0/308886625 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4009ea0 con 0x7f71c81556f0 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71bffff700 1 -- 172.21.15.118:0/308886625 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c400b860 con 0x7f71c81556f0 2024-08-18T13:40:08.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71bffff700 1 -- 172.21.15.118:0/308886625 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c400bc00 con 0x7f71c81556f0 2024-08-18T13:40:08.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 -- 172.21.15.118:0/308886625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c81556f0 msgr2=0x7f71c80c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/308886625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c81556f0 0x7f71c80c15a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71c40099b0 tx=0x7f71c4009250 comp rx=0 tx=0).stop 2024-08-18T13:40:08.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 -- 172.21.15.118:0/308886625 shutdown_connections 2024-08-18T13:40:08.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/308886625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/308886625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c81556f0 0x7f71c80c15a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 -- 172.21.15.118:0/308886625 >> 172.21.15.118:0/308886625 conn(0x7f71c806c020 msgr2=0x7f71c80c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 -- 172.21.15.118:0/308886625 shutdown_connections 2024-08-18T13:40:08.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.266+0000 7f71ce0aa700 1 -- 172.21.15.118:0/308886625 wait complete. 2024-08-18T13:40:08.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 Processor -- start 2024-08-18T13:40:08.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 -- start start 2024-08-18T13:40:08.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c80684e0 0x7f71c81506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c81556f0 0x7f71c814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.276 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8148990 con 0x7f71c80684e0 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c814ee70 con 0x7f71c81556f0 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71ce0aa700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8152ba0 con 0x7f71c81561d0 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71cd0a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c80684e0 0x7f71c81506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71cd0a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c80684e0 0x7f71c81506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58568/0 (socket says 172.21.15.118:58568) 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2319096687 learned_addr learned my addr 172.21.15.118:0/2319096687 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c81556f0 0x7f71c814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.270+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 msgr2=0x7f71c8143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c81556f0 msgr2=0x7f71c814cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c81556f0 0x7f71c814cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2319096687 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c0009a20 con 0x7f71c80684e0 2024-08-18T13:40:08.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c80684e0 0x7f71c81506b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71c805eb30 tx=0x7f71c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71bd7fa700 1 -- 172.21.15.118:0/2319096687 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4000d10 con 0x7f71c80684e0 2024-08-18T13:40:08.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71bd7fa700 1 -- 172.21.15.118:0/2319096687 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c400bbe0 con 0x7f71c80684e0 2024-08-18T13:40:08.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c40094f0 con 0x7f71c80684e0 2024-08-18T13:40:08.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71bd7fa700 1 -- 172.21.15.118:0/2319096687 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4011690 con 0x7f71c80684e0 2024-08-18T13:40:08.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c814d8d0 con 0x7f71c80684e0 2024-08-18T13:40:08.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.274+0000 7f71bd7fa700 1 -- 172.21.15.118:0/2319096687 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f71c4011ac0 con 0x7f71c80684e0 2024-08-18T13:40:08.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71bd7fa700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f71ac060890 0x7f71ac062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71bd7fa700 1 -- 172.21.15.118:0/2319096687 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f71c407e5d0 con 0x7f71c80684e0 2024-08-18T13:40:08.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f71ac060890 0x7f71ac062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.282 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f71ac060890 0x7f71ac062d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f71c81530d0 tx=0x7f71b8007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 Processor -- start 2024-08-18T13:40:08.283 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 -- start start 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x12dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x12dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x12dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x1254ba0 con 0x10a8ae0 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x1250b40 con 0x10f3790 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x12160a0 con 0x10a8710 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71cd0a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x12dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71cd0a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x12dd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58572/0 (socket says 172.21.15.118:58572) 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.278+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2257657302 learned_addr learned my addr 172.21.15.118:0/2257657302 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x12dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x12dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 msgr2=0x12dc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x12dc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 msgr2=0x12dc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x12dc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cd0a8700 1 -- 172.21.15.118:0/2257657302 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10f49e0 con 0x10a8ae0 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x12dd2a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71c4014040 tx=0x7f71c407c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7133b31fad9ca1f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71b27fc700 1 -- 172.21.15.118:0/2257657302 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4052b30 con 0x10a8ae0 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71b27fc700 1 -- 172.21.15.118:0/2257657302 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4055680 con 0x10a8ae0 2024-08-18T13:40:08.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71b27fc700 1 -- 172.21.15.118:0/2257657302 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40593d0 con 0x10a8ae0 2024-08-18T13:40:08.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 -- 172.21.15.118:0/2257657302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 msgr2=0x12dd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x12dd2a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71c4014040 tx=0x7f71c407c760 comp rx=0 tx=0).stop 2024-08-18T13:40:08.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 -- 172.21.15.118:0/2257657302 shutdown_connections 2024-08-18T13:40:08.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x12dc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x12dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.288 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 --2- 172.21.15.118:0/2257657302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x12dd2a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 -- 172.21.15.118:0/2257657302 >> 172.21.15.118:0/2257657302 conn(0x10a8eb0 msgr2=0x131b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 -- 172.21.15.118:0/2257657302 shutdown_connections 2024-08-18T13:40:08.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 -- 172.21.15.118:0/2257657302 wait complete. 2024-08-18T13:40:08.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 -- start start 2024-08-18T13:40:08.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.282+0000 7f71d15bb740 1 Processor -- start 2024-08-18T13:40:08.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1325850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8ae0 0x1325d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x12e0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x1215d40 con 0x10f3790 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x11ee970 con 0x10a8710 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x1254640 con 0x10a8ae0 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 10 client.0 _ll_register_callbacks cb 0x11f2f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x12e0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x12e0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58578/0 (socket says 172.21.15.118:58578) 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 -- 172.21.15.118:0/1757592676 learned_addr learned my addr 172.21.15.118:0/1757592676 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8ae0 0x1325d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1325850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 -- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8ae0 msgr2=0x1325d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8ae0 0x1325d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 -- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 msgr2=0x1325850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1325850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 -- 172.21.15.118:0/1757592676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c814d8d0 con 0x10f3790 2024-08-18T13:40:08.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x12e0030 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71c0007fc0 tx=0x7f71c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627c4f1f146d693a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-08-18T13:40:08.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71a77fe700 1 -- 172.21.15.118:0/1757592676 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000e040 con 0x10f3790 2024-08-18T13:40:08.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71a77fe700 1 -- 172.21.15.118:0/1757592676 <== mon.0 v2:172.21.15.118:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f71c0013ae0 con 0x10f3790 2024-08-18T13:40:08.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10b6260 con 0x10f3790 2024-08-18T13:40:08.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71a77fe700 1 -- 172.21.15.118:0/1757592676 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c0013eb0 con 0x10f3790 2024-08-18T13:40:08.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 10 client.4758 fetch_fsmap learned FSMap version 9 2024-08-18T13:40:08.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x121e370 con 0x10f3790 2024-08-18T13:40:08.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71a77fe700 1 -- 172.21.15.118:0/1757592676 <== mon.0 v2:172.21.15.118:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10b6260 con 0x10f3790 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71a77fe700 1 -- 172.21.15.118:0/1757592676 <== mon.0 v2:172.21.15.118:3300/0 5 ==== fsmap(e 9) v1 ==== 4198+0+0 (secure 0 0 0) 0x7f71c0021440 con 0x10f3790 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 10 client.4758 fetch_fsmap finished waiting for FSMap version 9 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 10 client.4758 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 20 client.4758 populate_metadata read hostname 'smithi118' 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x132eed0 0x13312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 4 client.4758 mds_command: new command op to 4312 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 --> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x11004e0 con 0x132eed0 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.286+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x132eed0 0x13312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x132eed0 0x13312b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18db396b5dc0f6b9 server_cookie=53fbd646a573609f in_seq=0 out_seq=0 2024-08-18T13:40:08.295 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71a77fe700 10 client.4758 ms_handle_connect on v2:172.21.15.178:6836/2503344632 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71a77fe700 1 -- 172.21.15.118:0/1757592676 <== mds.0 v2:172.21.15.178:6836/2503344632 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x11004e0 con 0x132eed0 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71a77fe700 10 client.4758 handle_command_reply: tid=0 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 client.4758 shutdown 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 msgr2=0x12e0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x12e0030 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71c0007fc0 tx=0x7f71c000b040 comp rx=0 tx=0).stop 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 shutdown_connections 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x132eed0 0x13312b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x12e0030 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71a77fe700 0 client.4758 ms_handle_reset on v2:172.21.15.178:6836/2503344632 2024-08-18T13:40:08.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8ae0 0x1325d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1757592676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1325850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 >> 172.21.15.118:0/1757592676 conn(0x10a8eb0 msgr2=0x1319a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 shutdown_connections 2024-08-18T13:40:08.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 1 -- 172.21.15.118:0/1757592676 wait complete. 2024-08-18T13:40:08.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.290+0000 7f71d15bb740 20 client.4758 trim_cache size 0 max 16384 2024-08-18T13:40:08.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 Processor -- start 2024-08-18T13:40:08.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 -- start start 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x1324240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x1324780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x1371ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x124b080 con 0x10a8ae0 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x1228570 con 0x10f3790 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x1286bb0 con 0x10a8710 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71cc8a7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x1324780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71cc8a7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x1324780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58588/0 (socket says 172.21.15.118:58588) 2024-08-18T13:40:08.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71cc8a7700 1 -- 172.21.15.118:0/1986319522 learned_addr learned my addr 172.21.15.118:0/1986319522 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x1324240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.310+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x1371ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 -- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 msgr2=0x1324240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x1324240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 -- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 msgr2=0x1371ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x1371ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 -- 172.21.15.118:0/1986319522 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1204d80 con 0x10a8ae0 2024-08-18T13:40:08.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x1324240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:08.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x1324780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f71b8002c20 tx=0x7f71b8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a307c8539bc5570e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71b2ffd700 1 -- 172.21.15.118:0/1986319522 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8014860 con 0x10a8ae0 2024-08-18T13:40:08.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71b2ffd700 1 -- 172.21.15.118:0/1986319522 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b80149c0 con 0x10a8ae0 2024-08-18T13:40:08.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71b2ffd700 1 -- 172.21.15.118:0/1986319522 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8014d40 con 0x10a8ae0 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- 172.21.15.118:0/1986319522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 msgr2=0x1324780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x1324780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f71b8002c20 tx=0x7f71b8015000 comp rx=0 tx=0).stop 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- 172.21.15.118:0/1986319522 shutdown_connections 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10f3790 0x1371ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10a8ae0 0x1324780 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 --2- 172.21.15.118:0/1986319522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x10a8710 0x1324240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- 172.21.15.118:0/1986319522 >> 172.21.15.118:0/1986319522 conn(0x10a8eb0 msgr2=0x1323b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- 172.21.15.118:0/1986319522 shutdown_connections 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- 172.21.15.118:0/1986319522 wait complete. 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- start start 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 Processor -- start 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1376ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x133b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x133b640 0x133ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x124f9d0 con 0x10f3790 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x1125650 con 0x10a8710 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x11ff140 con 0x133b640 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71d15bb740 10 client.0 _ll_register_callbacks cb 0x1177d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x133b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cd0a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1376ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x133b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58592/0 (socket says 172.21.15.118:58592) 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 -- 172.21.15.118:0/3918341358 learned_addr learned my addr 172.21.15.118:0/3918341358 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cd8a9700 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x133b640 0x133ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 -- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x133b640 msgr2=0x133ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x133b640 0x133ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 -- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 msgr2=0x1376ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1376ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.314+0000 7f71cc8a7700 1 -- 172.21.15.118:0/3918341358 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b8002950 con 0x10f3790 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71cc8a7700 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x133b100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71b8003380 tx=0x7f71b8024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266e7633a26b301b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 1 -- 172.21.15.118:0/3918341358 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8033070 con 0x10f3790 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 1 -- 172.21.15.118:0/3918341358 <== mon.0 v2:172.21.15.118:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f71b801c040 con 0x10f3790 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71c4055680 con 0x10f3790 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 1 -- 172.21.15.118:0/3918341358 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71b80147e0 con 0x10f3790 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 10 client.4764 fetch_fsmap learned FSMap version 9 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x11f9130 con 0x10f3790 2024-08-18T13:40:08.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 1 -- 172.21.15.118:0/3918341358 <== mon.0 v2:172.21.15.118:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4055680 con 0x10f3790 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 1 -- 172.21.15.118:0/3918341358 <== mon.0 v2:172.21.15.118:3300/0 5 ==== fsmap(e 9) v1 ==== 4198+0+0 (secure 0 0 0) 0x7f71b8006820 con 0x10f3790 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 10 client.4764 fetch_fsmap finished waiting for FSMap version 9 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 10 client.4764 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 20 client.4764 populate_metadata read hostname 'smithi118' 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x1387ac0 0x1389ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 4 client.4764 mds_command: new command op to 4312 tid=0[{"prefix": "status"}] 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 --> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1125260 con 0x1387ac0 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x1387ac0 0x1389ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 10 client.4764 ms_handle_connect on v2:172.21.15.178:6836/2503344632 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71cd0a8700 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x1387ac0 0x1389ea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48241db055a2f578 server_cookie=41ba3d9e4d5ef3b0 in_seq=0 out_seq=0 2024-08-18T13:40:08.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 1 -- 172.21.15.118:0/3918341358 <== mds.0 v2:172.21.15.178:6836/2503344632 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x1125260 con 0x1387ac0 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 10 client.4764 handle_command_reply: tid=0 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 client.4764 shutdown 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 msgr2=0x133b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x133b100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71b8003380 tx=0x7f71b8024000 comp rx=0 tx=0).stop 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 shutdown_connections 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:6836/2503344632,v1:172.21.15.178:6837/2503344632] conn(0x1387ac0 0x1389ea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x133b640 0x133ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x10f3790 0x133b100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 --2- 172.21.15.118:0/3918341358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x10a8710 0x1376ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71a77fe700 0 client.4764 ms_handle_reset on v2:172.21.15.178:6836/2503344632 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 >> 172.21.15.118:0/3918341358 conn(0x10a8eb0 msgr2=0x13243c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 shutdown_connections 2024-08-18T13:40:08.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 1 -- 172.21.15.118:0/3918341358 wait complete. 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.318+0000 7f71d15bb740 20 client.4764 trim_cache size 0 max 16384 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_fsid": "ac1e0c0e-66a6-4fa9-9612-9b35e09366c7", 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout: "whoami": 0, 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4312, 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout: "want_state": "up:active", 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout: "state": "up:active", 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs", 2024-08-18T13:40:08.329 INFO:teuthology.orchestra.run.smithi118.stdout: "rank_uptime": 32.908653696000002, 2024-08-18T13:40:08.330 INFO:teuthology.orchestra.run.smithi118.stdout: "mdsmap_epoch": 9, 2024-08-18T13:40:08.330 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_epoch": 25, 2024-08-18T13:40:08.330 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_epoch_barrier": 25, 2024-08-18T13:40:08.330 INFO:teuthology.orchestra.run.smithi118.stdout: "uptime": 39.979692835000002 2024-08-18T13:40:08.330 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f71ac060890 msgr2=0x7f71ac062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f71ac060890 0x7f71ac062d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f71c81530d0 tx=0x7f71b8007ec0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c80684e0 msgr2=0x7f71c81506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c80684e0 0x7f71c81506b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71c805eb30 tx=0x7f71c4017000 comp rx=0 tx=0).stop 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 shutdown_connections 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f71ac060890 0x7f71ac062d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71c81561d0 0x7f71c8143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71c81556f0 0x7f71c814cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 --2- 172.21.15.118:0/2319096687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71c80684e0 0x7f71c81506b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 >> 172.21.15.118:0/2319096687 conn(0x7f71c806c020 msgr2=0x7f71c814a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 shutdown_connections 2024-08-18T13:40:08.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.322+0000 7f71ce0aa700 1 -- 172.21.15.118:0/2319096687 wait complete. 2024-08-18T13:40:08.339 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-08-18T13:40:08.349 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-08-18T13:40:08.349 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-08-18T13:40:08.350 INFO:tasks.ceph_fuse:client.0 config is {} 2024-08-18T13:40:08.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:08.350 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2024-08-18T13:40:08.356 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-08-18T13:40:08.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:08.356 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T13:40:08.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:08.368 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link delete ceph-brx 2024-08-18T13:40:08.555 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-08-18T13:40:08.555 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-18T13:40:08.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.666+0000 7f246ebd3700 1 Processor -- start 2024-08-18T13:40:08.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.666+0000 7f246ebd3700 1 -- start start 2024-08-18T13:40:08.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f246814cfe0 0x7f2468152f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f2468154650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246806a6f0 con 0x7f246806e1b0 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24680c0540 con 0x7f246814cfe0 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246806b830 con 0x7f246814d3b0 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f246814cfe0 0x7f2468152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f246814cfe0 0x7f2468152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58604/0 (socket says 172.21.15.118:58604) 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 -- 172.21.15.118:0/632744334 learned_addr learned my addr 172.21.15.118:0/632744334 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246d3d0700 1 --2- 172.21.15.118:0/632744334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f2468154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 -- 172.21.15.118:0/632744334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 msgr2=0x7f2468154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 --2- 172.21.15.118:0/632744334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f2468154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 -- 172.21.15.118:0/632744334 >> v1:172.21.15.178:6789/0 conn(0x7f246806e1b0 legacy=0x7f24680beaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 -- 172.21.15.118:0/632744334 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246806a940 con 0x7f246814cfe0 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246dbd1700 1 --2- 172.21.15.118:0/632744334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f2468152f10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24640099b0 tx=0x7f2464009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867230324badd9e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246cbcf700 1 -- 172.21.15.118:0/632744334 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464009ea0 con 0x7f246814cfe0 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246cbcf700 1 -- 172.21.15.118:0/632744334 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246400b860 con 0x7f246814cfe0 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246cbcf700 1 -- 172.21.15.118:0/632744334 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246400bc00 con 0x7f246814cfe0 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- 172.21.15.118:0/632744334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 msgr2=0x7f2468152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 --2- 172.21.15.118:0/632744334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f2468152f10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24640099b0 tx=0x7f2464009250 comp rx=0 tx=0).stop 2024-08-18T13:40:08.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- 172.21.15.118:0/632744334 shutdown_connections 2024-08-18T13:40:08.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 --2- 172.21.15.118:0/632744334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f2468154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 --2- 172.21.15.118:0/632744334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f2468152f10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- 172.21.15.118:0/632744334 >> 172.21.15.118:0/632744334 conn(0x7f246806d6a0 msgr2=0x7f246806daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- 172.21.15.118:0/632744334 shutdown_connections 2024-08-18T13:40:08.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.670+0000 7f246ebd3700 1 -- 172.21.15.118:0/632744334 wait complete. 2024-08-18T13:40:08.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 Processor -- start 2024-08-18T13:40:08.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 -- start start 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f246806e1b0 0x7f24680c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f24680c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f24680ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468150f10 con 0x7f246814cfe0 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468151440 con 0x7f246806e1b0 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24680c87b0 con 0x7f246814d3b0 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f24680c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246dbd1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f246806e1b0 0x7f24680c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f24680c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58606/0 (socket says 172.21.15.118:58606) 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 -- 172.21.15.118:0/150140170 learned_addr learned my addr 172.21.15.118:0/150140170 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246e3d2700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f24680ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 -- 172.21.15.118:0/150140170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 msgr2=0x7f24680ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f24680ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 -- 172.21.15.118:0/150140170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f246806e1b0 msgr2=0x7f24680c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f246806e1b0 0x7f24680c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 -- 172.21.15.118:0/150140170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24640094f0 con 0x7f246814cfe0 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246d3d0700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f24680c91b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f245800ec50 tx=0x7f24580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f245dffb700 1 -- 172.21.15.118:0/150140170 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458009690 con 0x7f246814cfe0 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f245dffb700 1 -- 172.21.15.118:0/150140170 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245800c850 con 0x7f246814cfe0 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24681671b0 con 0x7f246814cfe0 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f245dffb700 1 -- 172.21.15.118:0/150140170 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245800cbf0 con 0x7f246814cfe0 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.674+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24680c8440 con 0x7f246814cfe0 2024-08-18T13:40:08.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.678+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24680c8920 con 0x7f246814cfe0 2024-08-18T13:40:08.682 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.678+0000 7f245dffb700 1 -- 172.21.15.118:0/150140170 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2458013470 con 0x7f246814cfe0 2024-08-18T13:40:08.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.678+0000 7f245dffb700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f244c060890 0x7f244c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:08.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.678+0000 7f245dffb700 1 -- 172.21.15.118:0/150140170 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f245807cf80 con 0x7f246814cfe0 2024-08-18T13:40:08.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.678+0000 7f246dbd1700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f244c060890 0x7f244c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:08.684 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.678+0000 7f246dbd1700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f244c060890 0x7f244c062d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24640099b0 tx=0x7f2464017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:08.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.686+0000 7f245dffb700 1 -- 172.21.15.118:0/150140170 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f24580ad480 con 0x7f246814cfe0 2024-08-18T13:40:08.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.886+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24680713a0 con 0x7f246814cfe0 2024-08-18T13:40:08.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f245dffb700 1 -- 172.21.15.118:0/150140170 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2458017070 con 0x7f246814cfe0 2024-08-18T13:40:08.893 INFO:teuthology.orchestra.run.smithi118.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f244c060890 msgr2=0x7f244c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f244c060890 0x7f244c062d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24640099b0 tx=0x7f2464017000 comp rx=0 tx=0).stop 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 msgr2=0x7f24680c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f24680c91b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f245800ec50 tx=0x7f24580091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 shutdown_connections 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f244c060890 0x7f244c062d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f246814d3b0 0x7f24680ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f246814cfe0 0x7f24680c91b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 --2- 172.21.15.118:0/150140170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f246806e1b0 0x7f24680c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 >> 172.21.15.118:0/150140170 conn(0x7f246806d6a0 msgr2=0x7f24680c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 shutdown_connections 2024-08-18T13:40:08.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:08.890+0000 7f246ebd3700 1 -- 172.21.15.118:0/150140170 wait complete. 2024-08-18T13:40:08.907 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-18T13:40:08.907 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-18T13:40:08.907 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi118.front.sepia.ceph.com 2024-08-18T13:40:08.908 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-18T13:40:08.908 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:08.908 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-18T13:40:08.908 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-18T13:40:08.908 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-18T13:40:08.908 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-18T13:40:08.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:08.909 DEBUG:teuthology.orchestra.run.smithi118:> ip addr 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: inet 127.0.0.1/8 scope host lo 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 ::1/128 scope host 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-18T13:40:08.914 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cc brd ff:ff:ff:ff:ff:ff 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:d8:a4:cd brd ff:ff:ff:ff:ff:ff 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:24 brd ff:ff:ff:ff:ff:ff 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 0c:c4:7a:8f:d7:25 brd ff:ff:ff:ff:ff:ff 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.21.15.118/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft 42580sec preferred_lft 42580sec 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: link/ether 02:42:27:94:1d:74 brd ff:ff:ff:ff:ff:ff 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-18T13:40:08.915 INFO:teuthology.orchestra.run.smithi118.stdout: valid_lft forever preferred_lft forever 2024-08-18T13:40:08.916 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-18T13:40:08.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link add name ceph-brx type bridge 2024-08-18T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip addr flush dev ceph-brx 2024-08-18T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set ceph-brx up 2024-08-18T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-18T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:40:08.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.000 DEBUG:teuthology.orchestra.run.smithi118:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-18T13:40:09.009 INFO:teuthology.orchestra.run.smithi118.stdout:1 2024-08-18T13:40:09.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.010 DEBUG:teuthology.orchestra.run.smithi118:> route 2024-08-18T13:40:09.017 INFO:teuthology.orchestra.run.smithi118.stdout:Kernel IP routing table 2024-08-18T13:40:09.017 INFO:teuthology.orchestra.run.smithi118.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-18T13:40:09.017 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-18T13:40:09.017 INFO:teuthology.orchestra.run.smithi118.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-18T13:40:09.017 INFO:teuthology.orchestra.run.smithi118.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-18T13:40:09.017 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-18T13:40:09.018 INFO:teuthology.orchestra.run.smithi118.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-18T13:40:09.018 INFO:teuthology.orchestra.run.smithi118.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-18T13:40:09.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.018 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:40:09.018 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:40:09.018 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-18T13:40:09.018 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-18T13:40:09.018 DEBUG:teuthology.orchestra.run.smithi118:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-18T13:40:09.018 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:40:09.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.054 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2024-08-18T13:40:09.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.104 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list-id 2024-08-18T13:40:09.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.111 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:40:09.111 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:40:09.111 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-18T13:40:09.111 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-18T13:40:09.111 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:40:09.141 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-18T13:40:09.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-18T13:40:09.142 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:40:09.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:40:09.243 DEBUG:teuthology.orchestra.run.smithi118:> (cd / && exec bash -c ' 2024-08-18T13:40:09.244 DEBUG:teuthology.orchestra.run.smithi118:> set -e 2024-08-18T13:40:09.244 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set brx.0 up 2024-08-18T13:40:09.244 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link set dev brx.0 master ceph-brx 2024-08-18T13:40:09.244 DEBUG:teuthology.orchestra.run.smithi118:> ') 2024-08-18T13:40:09.270 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-18T13:40:09.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T13:40:09.270 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:09.276 INFO:teuthology.orchestra.run.smithi118.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-18T13:40:09.276 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T13:40:09.276 DEBUG:teuthology.orchestra.run.smithi118:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:09.323 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2024-08-18T13:40:09.339 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-18T13:40:09.339 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-18T13:40:09.349 INFO:teuthology.orchestra.run.smithi118.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-18T13:40:09.350 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-18T13:40:09.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:40:09.351 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2024-08-18T13:40:09.355 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-18T13:40:09.356 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-18T13:40:09.358 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2024-08-18T13:40:09.375 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-18T13:40:09.375 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-18T13:40:09.385 INFO:teuthology.orchestra.run.smithi118.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-18T13:40:09.386 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-18T13:40:09.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:40:09.387 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2024-08-18T13:40:09.443 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi118.stderr:2024-08-18T13:40:09.438+0000 7fe0314ed380 -1 init, newargv = 0x55e7a6d5bc90 newargc=15 2024-08-18T13:40:09.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi118.stderr:ceph-fuse[32128]: starting ceph client 2024-08-18T13:40:09.452 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi118.stderr:ceph-fuse[32128]: starting fuse 2024-08-18T13:40:10.493 DEBUG:teuthology.orchestra.run.smithi118:> sudo modprobe fuse 2024-08-18T13:40:10.511 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-18T13:40:10.512 DEBUG:teuthology.orchestra.run.smithi118:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-18T13:40:10.569 INFO:teuthology.orchestra.run.smithi118.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-18T13:40:10.570 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-18T13:40:10.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:40:10.570 DEBUG:teuthology.orchestra.run.smithi118:> ls /sys/fs/fuse/connections 2024-08-18T13:40:10.576 INFO:teuthology.orchestra.run.smithi118.stdout:52 2024-08-18T13:40:10.576 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> import glob 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> import re 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> import os 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> import subprocess 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> def _find_admin_socket(client_name): 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> files = glob.glob(asok_path) 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> # Given a non-glob path, it better be there 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> if "*" not in asok_path: 2024-08-18T13:40:10.577 DEBUG:teuthology.orchestra.run.smithi118:> assert(len(files) == 1) 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> return files[0] 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> for f in files: 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> contents = proc_f.read() 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> if mountpoint in contents: 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> return f 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> print(_find_admin_socket("client.0")) 2024-08-18T13:40:10.578 DEBUG:teuthology.orchestra.run.smithi118:> ' 2024-08-18T13:40:10.656 INFO:teuthology.orchestra.run.smithi118.stdout:/var/run/ceph/ceph-client.0.32128.asok 2024-08-18T13:40:10.843 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.32128.asok 2024-08-18T13:40:10.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:40:10.843 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.32128.asok status 2024-08-18T13:40:10.980 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "metadata": { 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_sha1": "a17c67c052c1476b20f99d847823755b26703128", 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 17.2.7-1375-ga17c67c0 (a17c67c052c1476b20f99d847823755b26703128) quincy (stable)", 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "entity_id": "0", 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "hostname": "smithi118", 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "pid": "32128", 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "root": "/" 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_count": 0, 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "dentry_pinned_count": 0, 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 4779, 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "inst": { 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "name": { 2024-08-18T13:40:10.981 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "client", 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "num": 4779 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2706152080 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": { 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "type": "v1", 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "addr": "192.168.0.1:0", 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "nonce": 2706152080 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-08-18T13:40:10.982 INFO:teuthology.orchestra.run.smithi118.stdout: "inst_str": "client.4779 192.168.0.1:0/2706152080", 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout: "addr_str": "192.168.0.1:0/2706152080", 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout: "inode_count": 1, 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_epoch": 9, 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch": 25, 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_epoch_barrier": 0, 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout: "blocklisted": false, 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs" 2024-08-18T13:40:10.983 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:40:10.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:40:10.990 DEBUG:teuthology.orchestra.run.smithi118:> stat --file-system '--printf=%T 2024-08-18T13:40:10.990 DEBUG:teuthology.orchestra.run.smithi118:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:10.996 INFO:teuthology.orchestra.run.smithi118.stdout:fuseblk 2024-08-18T13:40:10.996 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:10.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-18T13:40:10.996 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:11.010 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-08-18T13:40:11.022 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:40:11.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5126403700 1 Processor -- start 2024-08-18T13:40:11.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5126403700 1 -- start start 2024-08-18T13:40:11.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5126403700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5120159120 0x7f5120160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5126403700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5120155270 con 0x7f512005e8d0 2024-08-18T13:40:11.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5126403700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5120068440 con 0x7f51200c2150 2024-08-18T13:40:11.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5126403700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51200c41f0 con 0x7f5120159120 2024-08-18T13:40:11.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5124c00700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f512005e8d0 0x7f512006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41128/0 (socket says 172.21.15.118:41128) 2024-08-18T13:40:11.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5124c00700 1 -- 172.21.15.118:0/3034342645 learned_addr learned my addr 172.21.15.118:0/3034342645 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:11.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 --2- 172.21.15.118:0/3034342645 >> v2:172.21.15.178:3300/0 conn(0x7f5120159120 0x7f5120160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363806790 0 0) 0x7f5120155270 con 0x7f512005e8d0 2024-08-18T13:40:11.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510c003570 con 0x7f512005e8d0 2024-08-18T13:40:11.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402629448 0 0) 0x7f510c003570 con 0x7f512005e8d0 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 >> v2:172.21.15.178:3300/0 conn(0x7f5120159120 msgr2=0x7f5120160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 --2- 172.21.15.118:0/3034342645 >> v2:172.21.15.178:3300/0 conn(0x7f5120159120 0x7f5120160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 >> v1:172.21.15.178:6789/0 conn(0x7f51200c2150 legacy=0x7f5120067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512005ed30 con 0x7f512005e8d0 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f5110001470 con 0x7f512005e8d0 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 >> v1:172.21.15.118:6789/0 conn(0x7f512005e8d0 legacy=0x7f512006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510c003a60 0x7f510c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c004810 0x7f510c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f510c0051c0 0x7f510c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200c41f0 con 0x7f510c0051c0 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c00c0d0 con 0x7f510c003a60 2024-08-18T13:40:11.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c00c290 con 0x7f510c004810 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 --2- 172.21.15.118:0/3034342645 >> v2:172.21.15.178:3300/0 conn(0x7f5120159120 0x7f5120160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f510c0051c0 0x7f510c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125401700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510c003a60 0x7f510c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5124c00700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c004810 0x7f510c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 -- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c004810 msgr2=0x7f510c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c004810 0x7f510c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 -- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510c003a60 msgr2=0x7f510c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510c003a60 0x7f510c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 -- 172.21.15.118:0/3034342645 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c003570 con 0x7f510c0051c0 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.162+0000 7f5125c02700 1 -- 172.21.15.118:0/3034342645 reap_dead start 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5125c02700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f510c0051c0 0x7f510c00ba60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5118009db0 tx=0x7f51180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76f992a01a57d14 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118009740 con 0x7f510c0051c0 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511800e920 con 0x7f510c0051c0 2024-08-18T13:40:11.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5117fff700 1 -- 172.21.15.118:0/3034342645 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800eca0 con 0x7f510c0051c0 2024-08-18T13:40:11.172 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5126403700 1 -- 172.21.15.118:0/3034342645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f510c0051c0 msgr2=0x7f510c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.172 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5126403700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f510c0051c0 0x7f510c00ba60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5118009db0 tx=0x7f51180091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.172 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5126403700 1 -- 172.21.15.118:0/3034342645 shutdown_connections 2024-08-18T13:40:11.172 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5126403700 1 --2- 172.21.15.118:0/3034342645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f510c0051c0 0x7f510c00ba60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5126403700 1 -- 172.21.15.118:0/3034342645 >> 172.21.15.118:0/3034342645 conn(0x7f51200c81e0 msgr2=0x7f51200c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:11.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5126403700 1 -- 172.21.15.118:0/3034342645 shutdown_connections 2024-08-18T13:40:11.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.166+0000 7f5126403700 1 -- 172.21.15.118:0/3034342645 wait complete. 2024-08-18T13:40:11.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 Processor -- start 2024-08-18T13:40:11.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 -- start start 2024-08-18T13:40:11.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c0051c0 0x7f5120151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f512013d760 0x7f5120157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5120151bd0 0x7f5120152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5120150220 con 0x7f512013d760 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5120165950 con 0x7f5120151bd0 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512014f160 con 0x7f510c0051c0 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f512013d760 0x7f5120157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f512013d760 0x7f5120157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58622/0 (socket says 172.21.15.118:58622) 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5125401700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c0051c0 0x7f5120151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 -- 172.21.15.118:0/1317901676 learned_addr learned my addr 172.21.15.118:0/1317901676 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5125c02700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5120151bd0 0x7f5120152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 -- 172.21.15.118:0/1317901676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c0051c0 msgr2=0x7f5120151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c0051c0 0x7f5120151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 -- 172.21.15.118:0/1317901676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5120151bd0 msgr2=0x7f5120152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5120151bd0 0x7f5120152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 -- 172.21.15.118:0/1317901676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5120165040 con 0x7f512013d760 2024-08-18T13:40:11.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5124c00700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f512013d760 0x7f5120157030 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5110001c00 tx=0x7f5120155160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f51157fa700 1 -- 172.21.15.118:0/1317901676 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110018030 con 0x7f512013d760 2024-08-18T13:40:11.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f51157fa700 1 -- 172.21.15.118:0/1317901676 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511000f660 con 0x7f512013d760 2024-08-18T13:40:11.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5120150410 con 0x7f512013d760 2024-08-18T13:40:11.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f51157fa700 1 -- 172.21.15.118:0/1317901676 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511000fa00 con 0x7f512013d760 2024-08-18T13:40:11.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.170+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51201655e0 con 0x7f512013d760 2024-08-18T13:40:11.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.174+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51201657a0 con 0x7f512013d760 2024-08-18T13:40:11.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.174+0000 7f51157fa700 1 -- 172.21.15.118:0/1317901676 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f511001b070 con 0x7f512013d760 2024-08-18T13:40:11.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.174+0000 7f51157fa700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5104060890 0x7f5104062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.174+0000 7f51157fa700 1 -- 172.21.15.118:0/1317901676 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f5110019070 con 0x7f512013d760 2024-08-18T13:40:11.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.174+0000 7f5125401700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5104060890 0x7f5104062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.174+0000 7f5125401700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5104060890 0x7f5104062d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f511c001750 tx=0x7f511c001640 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.182+0000 7f51157fa700 1 -- 172.21.15.118:0/1317901676 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5110050b00 con 0x7f512013d760 2024-08-18T13:40:11.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.346+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5120165ae0 con 0x7f512013d760 2024-08-18T13:40:11.352 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.346+0000 7f51157fa700 1 -- 172.21.15.118:0/1317901676 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f5110050690 con 0x7f512013d760 2024-08-18T13:40:11.352 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:11.353 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:40:11.354 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5104060890 msgr2=0x7f5104062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5104060890 0x7f5104062d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f511c001750 tx=0x7f511c001640 comp rx=0 tx=0).stop 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f512013d760 msgr2=0x7f5120157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f512013d760 0x7f5120157030 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5110001c00 tx=0x7f5120155160 comp rx=0 tx=0).stop 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 shutdown_connections 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5104060890 0x7f5104062d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5120151bd0 0x7f5120152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f512013d760 0x7f5120157030 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 --2- 172.21.15.118:0/1317901676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f510c0051c0 0x7f5120151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 >> 172.21.15.118:0/1317901676 conn(0x7f51200c81e0 msgr2=0x7f51200c5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 shutdown_connections 2024-08-18T13:40:11.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.350+0000 7f5126403700 1 -- 172.21.15.118:0/1317901676 wait complete. 2024-08-18T13:40:11.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T13:39:21.246221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-18T13:39:32.826757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-18T13:39:33.731815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-18T13:39:37.447514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-18T13:40:11.367 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T13:40:11.381 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:40:11.383 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:40:11.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 Processor -- start 2024-08-18T13:40:11.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 -- start start 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f182006ac00 0x7f18200673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1820156280 0x7f1820152700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f18200c5110 0x7f1820152c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200c3f70 con 0x7f1820156280 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200c5ff0 con 0x7f18200c5110 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f1826afe700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18200c62d0 con 0x7f182006ac00 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f181ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1820156280 0x7f1820152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f181ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1820156280 0x7f1820152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58632/0 (socket says 172.21.15.118:58632) 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f181ffff700 1 -- 172.21.15.118:0/2932496850 learned_addr learned my addr 172.21.15.118:0/2932496850 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.466+0000 7f182489a700 1 --2- 172.21.15.118:0/2932496850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f182006ac00 0x7f18200673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f182509b700 1 --2- 172.21.15.118:0/2932496850 >> v2:172.21.15.178:3300/0 conn(0x7f18200c5110 0x7f1820152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181ffff700 1 -- 172.21.15.118:0/2932496850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f182006ac00 msgr2=0x7f18200673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181ffff700 1 --2- 172.21.15.118:0/2932496850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f182006ac00 0x7f18200673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181ffff700 1 -- 172.21.15.118:0/2932496850 >> v2:172.21.15.178:3300/0 conn(0x7f18200c5110 msgr2=0x7f1820152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181ffff700 1 --2- 172.21.15.118:0/2932496850 >> v2:172.21.15.178:3300/0 conn(0x7f18200c5110 0x7f1820152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181ffff700 1 -- 172.21.15.118:0/2932496850 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18200c3640 con 0x7f1820156280 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181ffff700 1 --2- 172.21.15.118:0/2932496850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1820156280 0x7f1820152700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1820070180 tx=0x7f1814009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca99e55550da29e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181f7fe700 1 -- 172.21.15.118:0/2932496850 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814009e70 con 0x7f1820156280 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181f7fe700 1 -- 172.21.15.118:0/2932496850 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181400b860 con 0x7f1820156280 2024-08-18T13:40:11.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f181f7fe700 1 -- 172.21.15.118:0/2932496850 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400bc20 con 0x7f1820156280 2024-08-18T13:40:11.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 -- 172.21.15.118:0/2932496850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1820156280 msgr2=0x7f1820152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 --2- 172.21.15.118:0/2932496850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1820156280 0x7f1820152700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1820070180 tx=0x7f1814009250 comp rx=0 tx=0).stop 2024-08-18T13:40:11.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 -- 172.21.15.118:0/2932496850 shutdown_connections 2024-08-18T13:40:11.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 --2- 172.21.15.118:0/2932496850 >> v2:172.21.15.178:3300/0 conn(0x7f18200c5110 0x7f1820152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 --2- 172.21.15.118:0/2932496850 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1820156280 0x7f1820152700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 --2- 172.21.15.118:0/2932496850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f182006ac00 0x7f18200673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 -- 172.21.15.118:0/2932496850 >> 172.21.15.118:0/2932496850 conn(0x7f18200c9670 msgr2=0x7f18200c5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:11.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 -- 172.21.15.118:0/2932496850 shutdown_connections 2024-08-18T13:40:11.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 -- 172.21.15.118:0/2932496850 wait complete. 2024-08-18T13:40:11.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 Processor -- start 2024-08-18T13:40:11.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 -- start start 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f182006ac00 0x7f18201657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18200c5110 0x7f182014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.470+0000 7f1826afe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1820156280 0x7f1820068610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f1826afe700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1820165d10 con 0x7f18200c5110 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f1826afe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1820154270 con 0x7f182006ac00 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f1826afe700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1820068180 con 0x7f1820156280 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18200c5110 0x7f182014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18200c5110 0x7f182014ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58636/0 (socket says 172.21.15.118:58636) 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 -- 172.21.15.118:0/4060747215 learned_addr learned my addr 172.21.15.118:0/4060747215 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f182509b700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1820156280 0x7f1820068610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f182489a700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f182006ac00 0x7f18201657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 -- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1820156280 msgr2=0x7f1820068610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1820156280 0x7f1820068610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 -- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f182006ac00 msgr2=0x7f18201657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f182006ac00 0x7f18201657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 -- 172.21.15.118:0/4060747215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140094f0 con 0x7f18200c5110 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f182489a700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f182006ac00 0x7f18201657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181ffff700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18200c5110 0x7f182014ca20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fd0 tx=0x7f1814017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181cff9700 1 -- 172.21.15.118:0/4060747215 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181401f070 con 0x7f18200c5110 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181cff9700 1 -- 172.21.15.118:0/4060747215 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1814002f30 con 0x7f18200c5110 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18201539d0 con 0x7f18200c5110 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181cff9700 1 -- 172.21.15.118:0/4060747215 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400b9d0 con 0x7f18200c5110 2024-08-18T13:40:11.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1820153f90 con 0x7f18200c5110 2024-08-18T13:40:11.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18200648f0 con 0x7f18200c5110 2024-08-18T13:40:11.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.474+0000 7f181cff9700 1 -- 172.21.15.118:0/4060747215 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1814023070 con 0x7f18200c5110 2024-08-18T13:40:11.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.478+0000 7f181cff9700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f180c060890 0x7f180c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.478+0000 7f181cff9700 1 -- 172.21.15.118:0/4060747215 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f181407dfd0 con 0x7f18200c5110 2024-08-18T13:40:11.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.478+0000 7f182489a700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f180c060890 0x7f180c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.478+0000 7f182489a700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f180c060890 0x7f180c062d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1810009ce0 tx=0x7f1810008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.482+0000 7f181cff9700 1 -- 172.21.15.118:0/4060747215 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f181404e1e0 con 0x7f18200c5110 2024-08-18T13:40:11.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.650+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18200677e0 con 0x7f18200c5110 2024-08-18T13:40:11.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.650+0000 7f181cff9700 1 -- 172.21.15.118:0/4060747215 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1814015020 con 0x7f18200c5110 2024-08-18T13:40:11.655 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f180c060890 msgr2=0x7f180c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f180c060890 0x7f180c062d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1810009ce0 tx=0x7f1810008040 comp rx=0 tx=0).stop 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18200c5110 msgr2=0x7f182014ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18200c5110 0x7f182014ca20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fd0 tx=0x7f1814017000 comp rx=0 tx=0).stop 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 shutdown_connections 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f180c060890 0x7f180c062d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1820156280 0x7f1820068610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f18200c5110 0x7f182014ca20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 --2- 172.21.15.118:0/4060747215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f182006ac00 0x7f18201657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 >> 172.21.15.118:0/4060747215 conn(0x7f18200c9670 msgr2=0x7f18200c2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:11.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 shutdown_connections 2024-08-18T13:40:11.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.654+0000 7f1826afe700 1 -- 172.21.15.118:0/4060747215 wait complete. 2024-08-18T13:40:11.669 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-18T13:40:11.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.774+0000 7ff4bc38e700 1 Processor -- start 2024-08-18T13:40:11.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.774+0000 7ff4bc38e700 1 -- start start 2024-08-18T13:40:11.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff4b40c4670 0x7ff4b406c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b406bdd0 con 0x7ff4b40bf270 2024-08-18T13:40:11.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b4153ff0 con 0x7ff4b40c4670 2024-08-18T13:40:11.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40cab90 con 0x7ff4b40beea0 2024-08-18T13:40:11.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff4b40c4670 0x7ff4b406c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff4b40c4670 0x7ff4b406c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58642/0 (socket says 172.21.15.118:58642) 2024-08-18T13:40:11.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 -- 172.21.15.118:0/1520670535 learned_addr learned my addr 172.21.15.118:0/1520670535 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:11.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4ba12a700 1 --2- 172.21.15.118:0/1520670535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 -- 172.21.15.118:0/1520670535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 msgr2=0x7ff4b4153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:11.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 --2- 172.21.15.118:0/1520670535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 -- 172.21.15.118:0/1520670535 >> v1:172.21.15.178:6789/0 conn(0x7ff4b40bf270 legacy=0x7ff4b4156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 -- 172.21.15.118:0/1520670535 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40bf8f0 con 0x7ff4b40c4670 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9929700 1 --2- 172.21.15.118:0/1520670535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b406c260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80099b0 tx=0x7ff4a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc323084b55935be server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9128700 1 -- 172.21.15.118:0/1520670535 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800b860 con 0x7ff4b40c4670 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9128700 1 -- 172.21.15.118:0/1520670535 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a800b9c0 con 0x7ff4b40c4670 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4b9128700 1 -- 172.21.15.118:0/1520670535 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800bd60 con 0x7ff4b40c4670 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1520670535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 msgr2=0x7ff4b406c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1520670535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b406c260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80099b0 tx=0x7ff4a8009250 comp rx=0 tx=0).stop 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1520670535 shutdown_connections 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1520670535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b406c260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1520670535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1520670535 >> 172.21.15.118:0/1520670535 conn(0x7ff4b406e270 msgr2=0x7ff4b40c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1520670535 shutdown_connections 2024-08-18T13:40:11.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.778+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1520670535 wait complete. 2024-08-18T13:40:11.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 Processor -- start 2024-08-18T13:40:11.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 -- start start 2024-08-18T13:40:11.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40bf270 0x7ff4b40c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b40c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40adbe0 con 0x7ff4b40c4670 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b414dff0 con 0x7ff4b40bf270 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4bc38e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40ad640 con 0x7ff4b40beea0 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b40c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b40c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58656/0 (socket says 172.21.15.118:58656) 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 -- 172.21.15.118:0/1498282316 learned_addr learned my addr 172.21.15.118:0/1498282316 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4b9929700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40bf270 0x7ff4b40c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba12a700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 -- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 msgr2=0x7ff4b4149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 -- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40bf270 msgr2=0x7ff4b40c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40bf270 0x7ff4b40c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 -- 172.21.15.118:0/1498282316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80094f0 con 0x7ff4b40c4670 2024-08-18T13:40:11.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.782+0000 7ff4ba92b700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b40c2af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c910 tx=0x7ff4ac001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4a67fc700 1 -- 172.21.15.118:0/1498282316 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac012630 con 0x7ff4b40c4670 2024-08-18T13:40:11.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4a67fc700 1 -- 172.21.15.118:0/1498282316 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac012790 con 0x7ff4b40c4670 2024-08-18T13:40:11.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b40ad900 con 0x7ff4b40c4670 2024-08-18T13:40:11.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4a67fc700 1 -- 172.21.15.118:0/1498282316 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac012b50 con 0x7ff4b40c4670 2024-08-18T13:40:11.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b414dc80 con 0x7ff4b40c4670 2024-08-18T13:40:11.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40648f0 con 0x7ff4b40c4670 2024-08-18T13:40:11.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4a67fc700 1 -- 172.21.15.118:0/1498282316 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff4ac01a3a0 con 0x7ff4b40c4670 2024-08-18T13:40:11.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4a67fc700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff498060860 0x7ff498062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:11.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4a67fc700 1 -- 172.21.15.118:0/1498282316 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7ff4ac07d110 con 0x7ff4b40c4670 2024-08-18T13:40:11.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.786+0000 7ff4ba12a700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff498060860 0x7ff498062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:11.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.790+0000 7ff4ba12a700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff498060860 0x7ff498062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006fd0 tx=0x7ff4b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:11.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.794+0000 7ff4a67fc700 1 -- 172.21.15.118:0/1498282316 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff4ac04d270 con 0x7ff4b40c4670 2024-08-18T13:40:11.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.954+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4b414e130 con 0x7ff4b40c4670 2024-08-18T13:40:11.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.954+0000 7ff4a67fc700 1 -- 172.21.15.118:0/1498282316 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff4ac050890 con 0x7ff4b40c4670 2024-08-18T13:40:11.959 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:11.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff498060860 msgr2=0x7ff498062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff498060860 0x7ff498062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006fd0 tx=0x7ff4b0008040 comp rx=0 tx=0).stop 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 msgr2=0x7ff4b40c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b40c2af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c910 tx=0x7ff4ac001710 comp rx=0 tx=0).stop 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 shutdown_connections 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff498060860 0x7ff498062d10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4b40c4670 0x7ff4b40c2af0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4b40bf270 0x7ff4b40c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 --2- 172.21.15.118:0/1498282316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4b40beea0 0x7ff4b4149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 >> 172.21.15.118:0/1498282316 conn(0x7ff4b406e270 msgr2=0x7ff4b40c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 shutdown_connections 2024-08-18T13:40:11.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:11.958+0000 7ff4bc38e700 1 -- 172.21.15.118:0/1498282316 wait complete. 2024-08-18T13:40:11.981 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-18T13:40:12.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 Processor -- start 2024-08-18T13:40:12.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 -- start start 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f06d0156160 0x7f06d0154e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d006a6f0 0x7f06d0155370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f06d01567c0 0x7f06d00bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d01546c0 con 0x7f06d01567c0 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d01549a0 con 0x7f06d0156160 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d65c7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0156530 con 0x7f06d006a6f0 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f06d01567c0 0x7f06d00bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f06d01567c0 0x7f06d00bea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58664/0 (socket says 172.21.15.118:58664) 2024-08-18T13:40:12.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 -- 172.21.15.118:0/1028603781 learned_addr learned my addr 172.21.15.118:0/1028603781 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:12.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d4dc4700 1 --2- 172.21.15.118:0/1028603781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d006a6f0 0x7f06d0155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d55c5700 1 --2- 172.21.15.118:0/1028603781 >> v2:172.21.15.178:3300/0 conn(0x7f06d0156160 0x7f06d0154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 -- 172.21.15.118:0/1028603781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d006a6f0 msgr2=0x7f06d0155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:12.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 --2- 172.21.15.118:0/1028603781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d006a6f0 0x7f06d0155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 -- 172.21.15.118:0/1028603781 >> v2:172.21.15.178:3300/0 conn(0x7f06d0156160 msgr2=0x7f06d0154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 --2- 172.21.15.118:0/1028603781 >> v2:172.21.15.178:3300/0 conn(0x7f06d0156160 0x7f06d0154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 -- 172.21.15.118:0/1028603781 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d006abe0 con 0x7f06d01567c0 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.086+0000 7f06d5dc6700 1 --2- 172.21.15.118:0/1028603781 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d01567c0 0x7f06d00bea80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f06d0070eb0 tx=0x7f06c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b192d132f5805514 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06c7fff700 1 -- 172.21.15.118:0/1028603781 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c800b6d0 con 0x7f06d01567c0 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06c7fff700 1 -- 172.21.15.118:0/1028603781 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c800b830 con 0x7f06d01567c0 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06c7fff700 1 -- 172.21.15.118:0/1028603781 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c800bbf0 con 0x7f06d01567c0 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 -- 172.21.15.118:0/1028603781 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d01567c0 msgr2=0x7f06d00bea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 --2- 172.21.15.118:0/1028603781 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d01567c0 0x7f06d00bea80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f06d0070eb0 tx=0x7f06c8009250 comp rx=0 tx=0).stop 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 -- 172.21.15.118:0/1028603781 shutdown_connections 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 --2- 172.21.15.118:0/1028603781 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d01567c0 0x7f06d00bea80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 --2- 172.21.15.118:0/1028603781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d006a6f0 0x7f06d0155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 --2- 172.21.15.118:0/1028603781 >> v2:172.21.15.178:3300/0 conn(0x7f06d0156160 0x7f06d0154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 -- 172.21.15.118:0/1028603781 >> 172.21.15.118:0/1028603781 conn(0x7f06d006d630 msgr2=0x7f06d006da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:12.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 -- 172.21.15.118:0/1028603781 shutdown_connections 2024-08-18T13:40:12.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.090+0000 7f06d65c7700 1 -- 172.21.15.118:0/1028603781 wait complete. 2024-08-18T13:40:12.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 Processor -- start 2024-08-18T13:40:12.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 -- start start 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d006a6f0 0x7f06d0167330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06d0156160 0x7f06d0148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d01567c0 0x7f06d01515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0164350 con 0x7f06d006a6f0 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0148f60 con 0x7f06d0156160 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0163740 con 0x7f06d01567c0 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d006a6f0 0x7f06d0167330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d006a6f0 0x7f06d0167330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58680/0 (socket says 172.21.15.118:58680) 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 -- 172.21.15.118:0/41778364 learned_addr learned my addr 172.21.15.118:0/41778364 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d5dc6700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d01567c0 0x7f06d01515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d4dc4700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06d0156160 0x7f06d0148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 -- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d01567c0 msgr2=0x7f06d01515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d01567c0 0x7f06d01515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 -- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06d0156160 msgr2=0x7f06d0148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06d0156160 0x7f06d0148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 -- 172.21.15.118:0/41778364 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80094f0 con 0x7f06d006a6f0 2024-08-18T13:40:12.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d55c5700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d006a6f0 0x7f06d0167330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009ef0 tx=0x7f06c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06c57fa700 1 -- 172.21.15.118:0/41778364 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c000cb70 con 0x7f06d006a6f0 2024-08-18T13:40:12.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06c57fa700 1 -- 172.21.15.118:0/41778364 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c000f040 con 0x7f06d006a6f0 2024-08-18T13:40:12.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d0163a20 con 0x7f06d006a6f0 2024-08-18T13:40:12.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06c57fa700 1 -- 172.21.15.118:0/41778364 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0014480 con 0x7f06d006a6f0 2024-08-18T13:40:12.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.094+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d0163fe0 con 0x7f06d006a6f0 2024-08-18T13:40:12.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.098+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d00c5cf0 con 0x7f06d006a6f0 2024-08-18T13:40:12.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.098+0000 7f06c57fa700 1 -- 172.21.15.118:0/41778364 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f06c00147f0 con 0x7f06d006a6f0 2024-08-18T13:40:12.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.098+0000 7f06c57fa700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f06bc060890 0x7f06bc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.098+0000 7f06c57fa700 1 -- 172.21.15.118:0/41778364 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f06c007dab0 con 0x7f06d006a6f0 2024-08-18T13:40:12.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.098+0000 7f06d4dc4700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f06bc060890 0x7f06bc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.102+0000 7f06d4dc4700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f06bc060890 0x7f06bc062d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06d0064ca0 tx=0x7f06cc0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.106+0000 7f06c57fa700 1 -- 172.21.15.118:0/41778364 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f06d00c5cf0 con 0x7f06d006a6f0 2024-08-18T13:40:12.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.266+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06d00c5cf0 con 0x7f06d006a6f0 2024-08-18T13:40:12.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.270+0000 7f06c57fa700 1 -- 172.21.15.118:0/41778364 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06d00c5cf0 con 0x7f06d006a6f0 2024-08-18T13:40:12.273 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 64 2024-08-18T13:40:12.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.270+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f06bc060890 msgr2=0x7f06bc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.270+0000 7f06d65c7700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f06bc060890 0x7f06bc062d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06d0064ca0 tx=0x7f06cc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.270+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d006a6f0 msgr2=0x7f06d0167330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.277 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.270+0000 7f06d65c7700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d006a6f0 0x7f06d0167330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009ef0 tx=0x7f06c000b040 comp rx=0 tx=0).stop 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 shutdown_connections 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f06bc060890 0x7f06bc062d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06d01567c0 0x7f06d01515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06d0156160 0x7f06d0148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 --2- 172.21.15.118:0/41778364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f06d006a6f0 0x7f06d0167330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 >> 172.21.15.118:0/41778364 conn(0x7f06d006d630 msgr2=0x7f06d006da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 shutdown_connections 2024-08-18T13:40:12.278 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.274+0000 7f06d65c7700 1 -- 172.21.15.118:0/41778364 wait complete. 2024-08-18T13:40:12.292 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-18T13:40:12.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14462ac700 1 Processor -- start 2024-08-18T13:40:12.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14462ac700 1 -- start start 2024-08-18T13:40:12.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14462ac700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f14400c87c0 0x7f1440068e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14462ac700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400689e0 con 0x7f14400c8b90 2024-08-18T13:40:12.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14462ac700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1440155630 con 0x7f14401551a0 2024-08-18T13:40:12.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14462ac700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1440156190 con 0x7f14400c87c0 2024-08-18T13:40:12.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f1445aab700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f14400c8b90 0x7f144014ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41138/0 (socket says 172.21.15.118:41138) 2024-08-18T13:40:12.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f1445aab700 1 -- 172.21.15.118:0/1805384135 learned_addr learned my addr 172.21.15.118:0/1805384135 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:12.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14452aa700 1 --2- 172.21.15.118:0/1805384135 >> v2:172.21.15.178:3300/0 conn(0x7f14400c87c0 0x7f1440068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f1433fff700 1 -- 172.21.15.118:0/1805384135 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520566030 0 0) 0x7f14400689e0 con 0x7f14400c8b90 2024-08-18T13:40:12.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f1433fff700 1 -- 172.21.15.118:0/1805384135 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f142c003570 con 0x7f14400c8b90 2024-08-18T13:40:12.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14452aa700 1 -- 172.21.15.118:0/1805384135 >> v1:172.21.15.178:6789/0 conn(0x7f14401551a0 legacy=0x7f14400c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14452aa700 1 -- 172.21.15.118:0/1805384135 >> v1:172.21.15.118:6789/0 conn(0x7f14400c8b90 legacy=0x7f144014ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14452aa700 1 -- 172.21.15.118:0/1805384135 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14401558f0 con 0x7f14400c87c0 2024-08-18T13:40:12.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.398+0000 7f14452aa700 1 --2- 172.21.15.118:0/1805384135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c87c0 0x7f1440068e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f143c009e60 tx=0x7f143c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35217ffa802ce7b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f1433fff700 1 -- 172.21.15.118:0/1805384135 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c00e070 con 0x7f14400c87c0 2024-08-18T13:40:12.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f1433fff700 1 -- 172.21.15.118:0/1805384135 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c00bbf0 con 0x7f14400c87c0 2024-08-18T13:40:12.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f1433fff700 1 -- 172.21.15.118:0/1805384135 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c012430 con 0x7f14400c87c0 2024-08-18T13:40:12.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 -- 172.21.15.118:0/1805384135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c87c0 msgr2=0x7f1440068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 --2- 172.21.15.118:0/1805384135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c87c0 0x7f1440068e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f143c009e60 tx=0x7f143c009780 comp rx=0 tx=0).stop 2024-08-18T13:40:12.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 -- 172.21.15.118:0/1805384135 shutdown_connections 2024-08-18T13:40:12.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 --2- 172.21.15.118:0/1805384135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c87c0 0x7f1440068e70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 -- 172.21.15.118:0/1805384135 >> 172.21.15.118:0/1805384135 conn(0x7f14400673d0 msgr2=0x7f14400677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:12.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 -- 172.21.15.118:0/1805384135 shutdown_connections 2024-08-18T13:40:12.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 -- 172.21.15.118:0/1805384135 wait complete. 2024-08-18T13:40:12.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 Processor -- start 2024-08-18T13:40:12.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.402+0000 7f14462ac700 1 -- start start 2024-08-18T13:40:12.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14400c87c0 0x7f1440145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c8b90 0x7f14400cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14401551a0 0x7f14400cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400b0b80 con 0x7f14400c87c0 2024-08-18T13:40:12.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400c9a10 con 0x7f14400c8b90 2024-08-18T13:40:12.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14400c9cb0 con 0x7f14401551a0 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14400c87c0 0x7f1440145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14400c87c0 0x7f1440145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58688/0 (socket says 172.21.15.118:58688) 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 -- 172.21.15.118:0/1879274413 learned_addr learned my addr 172.21.15.118:0/1879274413 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f1445aab700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14401551a0 0x7f14400cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f1444aa9700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c8b90 0x7f14400cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 -- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14401551a0 msgr2=0x7f14400cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14401551a0 0x7f14400cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 -- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c8b90 msgr2=0x7f14400cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c8b90 0x7f14400cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 -- 172.21.15.118:0/1879274413 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c009a20 con 0x7f14400c87c0 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f1444aa9700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c8b90 0x7f14400cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:40:12.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14452aa700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14400c87c0 0x7f1440145880 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1440089510 tx=0x7f143c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14317fa700 1 -- 172.21.15.118:0/1879274413 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c012e60 con 0x7f14400c87c0 2024-08-18T13:40:12.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14317fa700 1 -- 172.21.15.118:0/1879274413 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c027470 con 0x7f14400c87c0 2024-08-18T13:40:12.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14400c9f50 con 0x7f14400c87c0 2024-08-18T13:40:12.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14317fa700 1 -- 172.21.15.118:0/1879274413 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c027810 con 0x7f14400c87c0 2024-08-18T13:40:12.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14400ca510 con 0x7f14400c87c0 2024-08-18T13:40:12.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400ca6d0 con 0x7f14400c87c0 2024-08-18T13:40:12.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.406+0000 7f14317fa700 1 -- 172.21.15.118:0/1879274413 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f143c024070 con 0x7f14400c87c0 2024-08-18T13:40:12.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.410+0000 7f14317fa700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1424060860 0x7f1424062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.410+0000 7f14317fa700 1 -- 172.21.15.118:0/1879274413 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f143c07f3f0 con 0x7f14400c87c0 2024-08-18T13:40:12.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.410+0000 7f1444aa9700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1424060860 0x7f1424062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.410+0000 7f1444aa9700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1424060860 0x7f1424062d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1440156f10 tx=0x7f1434009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.414+0000 7f14317fa700 1 -- 172.21.15.118:0/1879274413 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f143c04f550 con 0x7f14400c87c0 2024-08-18T13:40:12.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.578+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1440071410 con 0x7f14400c87c0 2024-08-18T13:40:12.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.578+0000 7f14317fa700 1 -- 172.21.15.118:0/1879274413 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f143c052b70 con 0x7f14400c87c0 2024-08-18T13:40:12.584 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:12.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1424060860 msgr2=0x7f1424062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1424060860 0x7f1424062d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1440156f10 tx=0x7f1434009040 comp rx=0 tx=0).stop 2024-08-18T13:40:12.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14400c87c0 msgr2=0x7f1440145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14400c87c0 0x7f1440145880 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1440089510 tx=0x7f143c019000 comp rx=0 tx=0).stop 2024-08-18T13:40:12.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 shutdown_connections 2024-08-18T13:40:12.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1424060860 0x7f1424062d10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f14401551a0 0x7f14400cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14400c8b90 0x7f14400cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 --2- 172.21.15.118:0/1879274413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14400c87c0 0x7f1440145880 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 >> 172.21.15.118:0/1879274413 conn(0x7f14400673d0 msgr2=0x7f14400c0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:12.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 shutdown_connections 2024-08-18T13:40:12.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.582+0000 7f14462ac700 1 -- 172.21.15.118:0/1879274413 wait complete. 2024-08-18T13:40:12.597 INFO:tasks.mds_thrash:mds thrasher using random seed: 1723988412 2024-08-18T13:40:12.597 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:40:12.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.694+0000 7f87428be700 1 Processor -- start 2024-08-18T13:40:12.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.694+0000 7f87428be700 1 -- start start 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f873c0be5a0 0x7f873c151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873c14bd10 0x7f873c152200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c06de00 con 0x7f873c14b940 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c06fac0 con 0x7f873c0be5a0 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c151870 con 0x7f873c14bd10 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f873c0be5a0 0x7f873c151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f873c0be5a0 0x7f873c151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58704/0 (socket says 172.21.15.118:58704) 2024-08-18T13:40:12.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87420bd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f873c14bd10 0x7f873c152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 -- 172.21.15.118:0/3022994776 learned_addr learned my addr 172.21.15.118:0/3022994776 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:12.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 -- 172.21.15.118:0/3022994776 >> v2:172.21.15.178:3300/0 conn(0x7f873c14bd10 msgr2=0x7f873c152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:12.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 --2- 172.21.15.118:0/3022994776 >> v2:172.21.15.178:3300/0 conn(0x7f873c14bd10 0x7f873c152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 -- 172.21.15.118:0/3022994776 >> v1:172.21.15.178:6789/0 conn(0x7f873c14b940 legacy=0x7f873c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 -- 172.21.15.118:0/3022994776 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0c2da0 con 0x7f873c0be5a0 2024-08-18T13:40:12.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87420bd700 1 --2- 172.21.15.118:0/3022994776 >> v2:172.21.15.178:3300/0 conn(0x7f873c14bd10 0x7f873c152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:12.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87410bb700 1 --2- 172.21.15.118:0/3022994776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c0be5a0 0x7f873c151cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f873c0701b0 tx=0x7f872c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c5a1204e95f2de server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87408ba700 1 -- 172.21.15.118:0/3022994776 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c009e70 con 0x7f873c0be5a0 2024-08-18T13:40:12.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87408ba700 1 -- 172.21.15.118:0/3022994776 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00b860 con 0x7f873c0be5a0 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87408ba700 1 -- 172.21.15.118:0/3022994776 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00bc00 con 0x7f873c0be5a0 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- 172.21.15.118:0/3022994776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c0be5a0 msgr2=0x7f873c151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 --2- 172.21.15.118:0/3022994776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c0be5a0 0x7f873c151cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f873c0701b0 tx=0x7f872c009250 comp rx=0 tx=0).stop 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- 172.21.15.118:0/3022994776 shutdown_connections 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 --2- 172.21.15.118:0/3022994776 >> v2:172.21.15.178:3300/0 conn(0x7f873c14bd10 0x7f873c152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 --2- 172.21.15.118:0/3022994776 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c0be5a0 0x7f873c151cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- 172.21.15.118:0/3022994776 >> 172.21.15.118:0/3022994776 conn(0x7f873c06c020 msgr2=0x7f873c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- 172.21.15.118:0/3022994776 shutdown_connections 2024-08-18T13:40:12.705 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.698+0000 7f87428be700 1 -- 172.21.15.118:0/3022994776 wait complete. 2024-08-18T13:40:12.706 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 Processor -- start 2024-08-18T13:40:12.706 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 -- start start 2024-08-18T13:40:12.707 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873c0be5a0 0x7f873c167020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.707 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c14b940 0x7f873c140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.707 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873c14bd10 0x7f873c140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.707 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c167c80 con 0x7f873c14b940 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c14d470 con 0x7f873c14bd10 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c14efd0 con 0x7f873c0be5a0 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c14b940 0x7f873c140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c14b940 0x7f873c140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58716/0 (socket says 172.21.15.118:58716) 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 -- 172.21.15.118:0/3486863079 learned_addr learned my addr 172.21.15.118:0/3486863079 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87418bc700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873c0be5a0 0x7f873c167020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87420bd700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873c14bd10 0x7f873c140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 -- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873c0be5a0 msgr2=0x7f873c167020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873c0be5a0 0x7f873c167020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 -- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873c14bd10 msgr2=0x7f873c140640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873c14bd10 0x7f873c140640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 -- 172.21.15.118:0/3486863079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738009420 con 0x7f873c14b940 2024-08-18T13:40:12.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87410bb700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c14b940 0x7f873c140100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f8731ffb700 1 -- 172.21.15.118:0/3486863079 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c000d10 con 0x7f873c14b940 2024-08-18T13:40:12.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f8731ffb700 1 -- 172.21.15.118:0/3486863079 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00bbe0 con 0x7f873c14b940 2024-08-18T13:40:12.710 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c0094f0 con 0x7f873c14b940 2024-08-18T13:40:12.710 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f8731ffb700 1 -- 172.21.15.118:0/3486863079 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c011670 con 0x7f873c14b940 2024-08-18T13:40:12.710 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.702+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c160fa0 con 0x7f873c14b940 2024-08-18T13:40:12.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.706+0000 7f8731ffb700 1 -- 172.21.15.118:0/3486863079 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f872c011a60 con 0x7f873c14b940 2024-08-18T13:40:12.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.706+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c0713a0 con 0x7f873c14b940 2024-08-18T13:40:12.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.706+0000 7f8731ffb700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8720060860 0x7f8720062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:12.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.706+0000 7f8731ffb700 1 -- 172.21.15.118:0/3486863079 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f872c07e300 con 0x7f873c14b940 2024-08-18T13:40:12.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.706+0000 7f87418bc700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8720060860 0x7f8720062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:12.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.706+0000 7f87418bc700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8720060860 0x7f8720062d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8734007fd0 tx=0x7f8734000f40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:12.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.714+0000 7f8731ffb700 1 -- 172.21.15.118:0/3486863079 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f872c04e8f0 con 0x7f873c14b940 2024-08-18T13:40:12.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.878+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f873c067bb0 con 0x7f873c14b940 2024-08-18T13:40:12.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f8731ffb700 1 -- 172.21.15.118:0/3486863079 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17239 (secure 0 0 0) 0x7f872c051b40 con 0x7f873c14b940 2024-08-18T13:40:12.886 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:12.887 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:39:39.788785+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:40:12.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8720060860 msgr2=0x7f8720062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8720060860 0x7f8720062d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8734007fd0 tx=0x7f8734000f40 comp rx=0 tx=0).stop 2024-08-18T13:40:12.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c14b940 msgr2=0x7f873c140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:12.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c14b940 0x7f873c140100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c017000 comp rx=0 tx=0).stop 2024-08-18T13:40:12.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 shutdown_connections 2024-08-18T13:40:12.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8720060860 0x7f8720062d10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873c14bd10 0x7f873c140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f873c14b940 0x7f873c140100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 --2- 172.21.15.118:0/3486863079 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f873c0be5a0 0x7f873c167020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:12.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 >> 172.21.15.118:0/3486863079 conn(0x7f873c06c020 msgr2=0x7f873c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:12.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.882+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 shutdown_connections 2024-08-18T13:40:12.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:12.886+0000 7f87428be700 1 -- 172.21.15.118:0/3486863079 wait complete. 2024-08-18T13:40:12.901 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T13:39:21.246221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-18T13:39:32.826757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-18T13:39:33.731815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-18T13:39:37.447514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-18T13:40:12.901 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T13:40:13.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb5388de700 1 Processor -- start 2024-08-18T13:40:13.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb5388de700 1 -- start start 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb5388de700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb534068820 0x7fb5340c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb5388de700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5340c8d40 0x7fb53414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb5388de700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5340c7980 con 0x7fb5340c0b00 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb5388de700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5340c2490 con 0x7fb534068820 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb5388de700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb534156280 con 0x7fb5340c8d40 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb532d9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fb5340c0b00 0x7fb5340c7e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41140/0 (socket says 172.21.15.118:41140) 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.010+0000 7fb532d9d700 1 -- 172.21.15.118:0/3892731235 learned_addr learned my addr 172.21.15.118:0/3892731235 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53359e700 1 --2- 172.21.15.118:0/3892731235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5340c8d40 0x7fb53414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53259c700 1 --2- 172.21.15.118:0/3892731235 >> v2:172.21.15.178:3300/0 conn(0x7fb534068820 0x7fb5340c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb531d9b700 1 -- 172.21.15.118:0/3892731235 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557545859 0 0) 0x7fb5340c7980 con 0x7fb5340c0b00 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb531d9b700 1 -- 172.21.15.118:0/3892731235 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb51c003570 con 0x7fb5340c0b00 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53359e700 1 -- 172.21.15.118:0/3892731235 >> v2:172.21.15.178:3300/0 conn(0x7fb534068820 msgr2=0x7fb5340c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53359e700 1 --2- 172.21.15.118:0/3892731235 >> v2:172.21.15.178:3300/0 conn(0x7fb534068820 0x7fb5340c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53359e700 1 -- 172.21.15.118:0/3892731235 >> v1:172.21.15.118:6789/0 conn(0x7fb5340c0b00 legacy=0x7fb5340c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53359e700 1 -- 172.21.15.118:0/3892731235 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534155ce0 con 0x7fb5340c8d40 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53259c700 1 --2- 172.21.15.118:0/3892731235 >> v2:172.21.15.178:3300/0 conn(0x7fb534068820 0x7fb5340c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb53359e700 1 --2- 172.21.15.118:0/3892731235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5340c8d40 0x7fb53414eb00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009e60 tx=0x7fb52c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52edb93b9ebb05b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb531d9b700 1 -- 172.21.15.118:0/3892731235 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00e070 con 0x7fb5340c8d40 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb531d9b700 1 -- 172.21.15.118:0/3892731235 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c00bab0 con 0x7fb5340c8d40 2024-08-18T13:40:13.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb531d9b700 1 -- 172.21.15.118:0/3892731235 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00be30 con 0x7fb5340c8d40 2024-08-18T13:40:13.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 -- 172.21.15.118:0/3892731235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5340c8d40 msgr2=0x7fb53414eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 --2- 172.21.15.118:0/3892731235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5340c8d40 0x7fb53414eb00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009e60 tx=0x7fb52c009780 comp rx=0 tx=0).stop 2024-08-18T13:40:13.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 -- 172.21.15.118:0/3892731235 shutdown_connections 2024-08-18T13:40:13.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 --2- 172.21.15.118:0/3892731235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5340c8d40 0x7fb53414eb00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 --2- 172.21.15.118:0/3892731235 >> v2:172.21.15.178:3300/0 conn(0x7fb534068820 0x7fb5340c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 -- 172.21.15.118:0/3892731235 >> 172.21.15.118:0/3892731235 conn(0x7fb5340cc630 msgr2=0x7fb5340c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:13.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 -- 172.21.15.118:0/3892731235 shutdown_connections 2024-08-18T13:40:13.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.014+0000 7fb5388de700 1 -- 172.21.15.118:0/3892731235 wait complete. 2024-08-18T13:40:13.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 Processor -- start 2024-08-18T13:40:13.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 -- start start 2024-08-18T13:40:13.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb534068820 0x7fb5340b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5340c0b00 0x7fb534165290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5340c8d40 0x7fb5341657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340aea40 con 0x7fb5340c8d40 2024-08-18T13:40:13.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340b0600 con 0x7fb5340c0b00 2024-08-18T13:40:13.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340afff0 con 0x7fb534068820 2024-08-18T13:40:13.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5340c8d40 0x7fb5341657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb532d9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb534068820 0x7fb5340b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb532d9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb534068820 0x7fb5340b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:46610/0 (socket says 172.21.15.118:46610) 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb532d9d700 1 -- 172.21.15.118:0/2042599368 learned_addr learned my addr 172.21.15.118:0/2042599368 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53259c700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5340c0b00 0x7fb534165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53359e700 1 -- 172.21.15.118:0/2042599368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb534068820 msgr2=0x7fb5340b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53359e700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb534068820 0x7fb5340b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53359e700 1 -- 172.21.15.118:0/2042599368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5340c0b00 msgr2=0x7fb534165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53359e700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5340c0b00 0x7fb534165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53359e700 1 -- 172.21.15.118:0/2042599368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528009420 con 0x7fb5340c8d40 2024-08-18T13:40:13.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb53359e700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5340c8d40 0x7fb5341657d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb53406fb80 tx=0x7fb52c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb51affd700 1 -- 172.21.15.118:0/2042599368 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c012dd0 con 0x7fb5340c8d40 2024-08-18T13:40:13.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb51affd700 1 -- 172.21.15.118:0/2042599368 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c027470 con 0x7fb5340c8d40 2024-08-18T13:40:13.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb51affd700 1 -- 172.21.15.118:0/2042599368 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c027830 con 0x7fb5340c8d40 2024-08-18T13:40:13.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c009a20 con 0x7fb5340c8d40 2024-08-18T13:40:13.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.018+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb534165e40 con 0x7fb5340c8d40 2024-08-18T13:40:13.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.022+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5341583b0 con 0x7fb5340c8d40 2024-08-18T13:40:13.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.022+0000 7fb51affd700 1 -- 172.21.15.118:0/2042599368 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb52c024070 con 0x7fb5340c8d40 2024-08-18T13:40:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.022+0000 7fb51affd700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb520060890 0x7fb520062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.022+0000 7fb51affd700 1 -- 172.21.15.118:0/2042599368 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fb52c07f4a0 con 0x7fb5340c8d40 2024-08-18T13:40:13.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.022+0000 7fb532d9d700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb520060890 0x7fb520062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.026+0000 7fb532d9d700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb520060890 0x7fb520062d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb524007fd0 tx=0x7fb524000bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.036 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.030+0000 7fb51affd700 1 -- 172.21.15.118:0/2042599368 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb5341583b0 con 0x7fb5340c8d40 2024-08-18T13:40:13.200 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.194+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5341583b0 con 0x7fb5340c8d40 2024-08-18T13:40:13.200 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.194+0000 7fb51affd700 1 -- 172.21.15.118:0/2042599368 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5341583b0 con 0x7fb5340c8d40 2024-08-18T13:40:13.201 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-08-18T13:40:13.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb520060890 msgr2=0x7fb520062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb520060890 0x7fb520062d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb524007fd0 tx=0x7fb524000bc0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5340c8d40 msgr2=0x7fb5341657d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5340c8d40 0x7fb5341657d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb53406fb80 tx=0x7fb52c019000 comp rx=0 tx=0).stop 2024-08-18T13:40:13.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 shutdown_connections 2024-08-18T13:40:13.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb520060890 0x7fb520062d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5340c8d40 0x7fb5341657d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5340c0b00 0x7fb534165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 --2- 172.21.15.118:0/2042599368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb534068820 0x7fb5340b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 >> 172.21.15.118:0/2042599368 conn(0x7fb5340cc630 msgr2=0x7fb5341550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:13.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 shutdown_connections 2024-08-18T13:40:13.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.198+0000 7fb5388de700 1 -- 172.21.15.118:0/2042599368 wait complete. 2024-08-18T13:40:13.215 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-18T13:40:13.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9eee5700 1 Processor -- start 2024-08-18T13:40:13.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9eee5700 1 -- start start 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9eee5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9e980c3350 0x7f9e9814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9eee5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9e9814be20 0x7f9e9815c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9eee5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e980c3de0 con 0x7f9e980c2f80 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9eee5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e980bfb90 con 0x7f9e980c3350 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9eee5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9806d620 con 0x7f9e9814be20 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9d6e2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9e980c3350 0x7f9e9814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9d6e2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9e980c3350 0x7f9e9814b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58724/0 (socket says 172.21.15.118:58724) 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9d6e2700 1 -- 172.21.15.118:0/3483426190 learned_addr learned my addr 172.21.15.118:0/3483426190 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:13.329 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.322+0000 7f9e9e6e4700 1 --2- 172.21.15.118:0/3483426190 >> v2:172.21.15.178:3300/0 conn(0x7f9e9814be20 0x7f9e9815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9d6e2700 1 -- 172.21.15.118:0/3483426190 >> v2:172.21.15.178:3300/0 conn(0x7f9e9814be20 msgr2=0x7f9e9815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:13.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9d6e2700 1 --2- 172.21.15.118:0/3483426190 >> v2:172.21.15.178:3300/0 conn(0x7f9e9814be20 0x7f9e9815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9d6e2700 1 -- 172.21.15.118:0/3483426190 >> v1:172.21.15.178:6789/0 conn(0x7f9e980c2f80 legacy=0x7f9e980c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9cee1700 1 -- 172.21.15.118:0/3483426190 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244758416 0 0) 0x7f9e980c3de0 con 0x7f9e980c2f80 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9d6e2700 1 -- 172.21.15.118:0/3483426190 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e980c4980 con 0x7f9e980c3350 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9e6e4700 1 --2- 172.21.15.118:0/3483426190 >> v2:172.21.15.178:3300/0 conn(0x7f9e9814be20 0x7f9e9815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9d6e2700 1 --2- 172.21.15.118:0/3483426190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c3350 0x7f9e9814b8e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e980c0920 tx=0x7f9e88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b980d6b08d2625b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9cee1700 1 -- 172.21.15.118:0/3483426190 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88009e70 con 0x7f9e980c3350 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9cee1700 1 -- 172.21.15.118:0/3483426190 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8800b860 con 0x7f9e980c3350 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9cee1700 1 -- 172.21.15.118:0/3483426190 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800bc00 con 0x7f9e980c3350 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 -- 172.21.15.118:0/3483426190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c3350 msgr2=0x7f9e9814b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/3483426190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c3350 0x7f9e9814b8e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e980c0920 tx=0x7f9e88009250 comp rx=0 tx=0).stop 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 -- 172.21.15.118:0/3483426190 shutdown_connections 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/3483426190 >> v2:172.21.15.178:3300/0 conn(0x7f9e9814be20 0x7f9e9815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/3483426190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c3350 0x7f9e9814b8e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 -- 172.21.15.118:0/3483426190 >> 172.21.15.118:0/3483426190 conn(0x7f9e9806c750 msgr2=0x7f9e980c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:13.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 -- 172.21.15.118:0/3483426190 shutdown_connections 2024-08-18T13:40:13.333 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 -- 172.21.15.118:0/3483426190 wait complete. 2024-08-18T13:40:13.333 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.326+0000 7f9e9eee5700 1 Processor -- start 2024-08-18T13:40:13.333 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 -- start start 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c2f80 0x7f9e9814ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e980c3350 0x7f9e980c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e9814be20 0x7f9e981662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9815da70 con 0x7f9e980c2f80 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e980c6b60 con 0x7f9e9814be20 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e980c6880 con 0x7f9e980c3350 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c2f80 0x7f9e9814ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c2f80 0x7f9e9814ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58726/0 (socket says 172.21.15.118:58726) 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 -- 172.21.15.118:0/2943665507 learned_addr learned my addr 172.21.15.118:0/2943665507 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9d6e2700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e980c3350 0x7f9e980c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9e6e4700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e9814be20 0x7f9e981662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 -- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e980c3350 msgr2=0x7f9e980c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e980c3350 0x7f9e980c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 -- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e9814be20 msgr2=0x7f9e981662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e9814be20 0x7f9e981662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 -- 172.21.15.118:0/2943665507 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90009420 con 0x7f9e980c2f80 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9dee3700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c2f80 0x7f9e9814ac70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c910 tx=0x7f9e94001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e8dffb700 1 -- 172.21.15.118:0/2943665507 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94006ba0 con 0x7f9e980c2f80 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e8dffb700 1 -- 172.21.15.118:0/2943665507 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94006d20 con 0x7f9e980c2f80 2024-08-18T13:40:13.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e880094f0 con 0x7f9e980c2f80 2024-08-18T13:40:13.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e8dffb700 1 -- 172.21.15.118:0/2943665507 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400aa30 con 0x7f9e980c2f80 2024-08-18T13:40:13.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9815d700 con 0x7f9e980c2f80 2024-08-18T13:40:13.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.330+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e98071410 con 0x7f9e980c2f80 2024-08-18T13:40:13.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.334+0000 7f9e8dffb700 1 -- 172.21.15.118:0/2943665507 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9e94012420 con 0x7f9e980c2f80 2024-08-18T13:40:13.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.334+0000 7f9e8dffb700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e84060860 0x7f9e84062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.334+0000 7f9e8dffb700 1 -- 172.21.15.118:0/2943665507 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f9e9407ce80 con 0x7f9e980c2f80 2024-08-18T13:40:13.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.334+0000 7f9e9d6e2700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e84060860 0x7f9e84062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.334+0000 7f9e9d6e2700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e84060860 0x7f9e84062d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e980c0920 tx=0x7f9e88017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.346 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.342+0000 7f9e8dffb700 1 -- 172.21.15.118:0/2943665507 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9e9404cf40 con 0x7f9e980c2f80 2024-08-18T13:40:13.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.506+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e980bf6c0 con 0x7f9e980c2f80 2024-08-18T13:40:13.510 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.506+0000 7f9e8dffb700 1 -- 172.21.15.118:0/2943665507 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e94050560 con 0x7f9e980c2f80 2024-08-18T13:40:13.510 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e84060860 msgr2=0x7f9e84062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e84060860 0x7f9e84062d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e980c0920 tx=0x7f9e88017000 comp rx=0 tx=0).stop 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c2f80 msgr2=0x7f9e9814ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c2f80 0x7f9e9814ac70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c910 tx=0x7f9e94001700 comp rx=0 tx=0).stop 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 shutdown_connections 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e84060860 0x7f9e84062d10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e9814be20 0x7f9e981662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e980c3350 0x7f9e980c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 --2- 172.21.15.118:0/2943665507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e980c2f80 0x7f9e9814ac70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 >> 172.21.15.118:0/2943665507 conn(0x7f9e9806c750 msgr2=0x7f9e980c4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:13.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 shutdown_connections 2024-08-18T13:40:13.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.510+0000 7f9e9eee5700 1 -- 172.21.15.118:0/2943665507 wait complete. 2024-08-18T13:40:13.530 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-18T13:40:13.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.638+0000 7f92b8956700 1 Processor -- start 2024-08-18T13:40:13.644 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.638+0000 7f92b8956700 1 -- start start 2024-08-18T13:40:13.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c0b00 0x7f92b40c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f92b40c8d40 0x7f92b414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b40c7980 con 0x7f92b4068820 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b40c2490 con 0x7f92b40c8d40 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40c15e0 con 0x7f92b40c0b00 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b2ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f92b4068820 0x7f92b40c0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41152/0 (socket says 172.21.15.118:41152) 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b2ffd700 1 -- 172.21.15.118:0/3820882705 learned_addr learned my addr 172.21.15.118:0/3820882705 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b3fff700 1 --2- 172.21.15.118:0/3820882705 >> v2:172.21.15.178:3300/0 conn(0x7f92b40c8d40 0x7f92b414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b37fe700 1 --2- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c0b00 0x7f92b40c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b27fc700 1 -- 172.21.15.118:0/3820882705 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836313166 0 0) 0x7f92b40c7980 con 0x7f92b4068820 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b27fc700 1 -- 172.21.15.118:0/3820882705 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a0003570 con 0x7f92b4068820 2024-08-18T13:40:13.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b3fff700 1 -- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c0b00 msgr2=0x7f92b40c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:13.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b3fff700 1 --2- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c0b00 0x7f92b40c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b3fff700 1 -- 172.21.15.118:0/3820882705 >> v1:172.21.15.118:6789/0 conn(0x7f92b4068820 legacy=0x7f92b40c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b3fff700 1 -- 172.21.15.118:0/3820882705 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4156280 con 0x7f92b40c8d40 2024-08-18T13:40:13.649 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b3fff700 1 --2- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c8d40 0x7f92b414eb00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009e60 tx=0x7f92ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73c56045778fa8a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.649 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b27fc700 1 -- 172.21.15.118:0/3820882705 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00e070 con 0x7f92b40c8d40 2024-08-18T13:40:13.649 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b27fc700 1 -- 172.21.15.118:0/3820882705 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac00bab0 con 0x7f92b40c8d40 2024-08-18T13:40:13.649 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b27fc700 1 -- 172.21.15.118:0/3820882705 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00be70 con 0x7f92b40c8d40 2024-08-18T13:40:13.649 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c8d40 msgr2=0x7f92b414eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.649 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 --2- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c8d40 0x7f92b414eb00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009e60 tx=0x7f92ac009780 comp rx=0 tx=0).stop 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- 172.21.15.118:0/3820882705 shutdown_connections 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 --2- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c8d40 0x7f92b414eb00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 --2- 172.21.15.118:0/3820882705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c0b00 0x7f92b40c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- 172.21.15.118:0/3820882705 >> 172.21.15.118:0/3820882705 conn(0x7f92b40cc630 msgr2=0x7f92b40c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- 172.21.15.118:0/3820882705 shutdown_connections 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.642+0000 7f92b8956700 1 -- 172.21.15.118:0/3820882705 wait complete. 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 Processor -- start 2024-08-18T13:40:13.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 -- start start 2024-08-18T13:40:13.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92b4068820 0x7f92b414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c0b00 0x7f92b414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c8d40 0x7f92b40c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b414db80 con 0x7f92b4068820 2024-08-18T13:40:13.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b414de70 con 0x7f92b40c0b00 2024-08-18T13:40:13.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b414e110 con 0x7f92b40c8d40 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92b4068820 0x7f92b414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92b4068820 0x7f92b414a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58738/0 (socket says 172.21.15.118:58738) 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 -- 172.21.15.118:0/2054386878 learned_addr learned my addr 172.21.15.118:0/2054386878 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b3fff700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c8d40 0x7f92b40c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b2ffd700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c0b00 0x7f92b414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 -- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c8d40 msgr2=0x7f92b40c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c8d40 0x7f92b40c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 -- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c0b00 msgr2=0x7f92b414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c0b00 0x7f92b414a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 -- 172.21.15.118:0/2054386878 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac009a20 con 0x7f92b4068820 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b37fe700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92b4068820 0x7f92b414a0a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f929c00ec80 tx=0x7f929c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92ab7fe700 1 -- 172.21.15.118:0/2054386878 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00c9d0 con 0x7f92b4068820 2024-08-18T13:40:13.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92ab7fe700 1 -- 172.21.15.118:0/2054386878 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c00cb30 con 0x7f92b4068820 2024-08-18T13:40:13.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b414d5c0 con 0x7f92b4068820 2024-08-18T13:40:13.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92ab7fe700 1 -- 172.21.15.118:0/2054386878 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c013470 con 0x7f92b4068820 2024-08-18T13:40:13.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.646+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b414f3e0 con 0x7f92b4068820 2024-08-18T13:40:13.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.650+0000 7f92ab7fe700 1 -- 172.21.15.118:0/2054386878 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f929c013820 con 0x7f92b4068820 2024-08-18T13:40:13.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.650+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b414d420 con 0x7f92b4068820 2024-08-18T13:40:13.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.650+0000 7f92ab7fe700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9294060860 0x7f9294062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.650+0000 7f92b2ffd700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9294060860 0x7f9294062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.650+0000 7f92ab7fe700 1 -- 172.21.15.118:0/2054386878 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f929c07d2f0 con 0x7f92b4068820 2024-08-18T13:40:13.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.650+0000 7f92b2ffd700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9294060860 0x7f9294062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f92b405e820 tx=0x7f92a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.658+0000 7f92ab7fe700 1 -- 172.21.15.118:0/2054386878 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f92b414d420 con 0x7f92b4068820 2024-08-18T13:40:13.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.822+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92b414d420 con 0x7f92b4068820 2024-08-18T13:40:13.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.822+0000 7f92ab7fe700 1 -- 172.21.15.118:0/2054386878 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92b414d420 con 0x7f92b4068820 2024-08-18T13:40:13.829 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 64 2024-08-18T13:40:13.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9294060860 msgr2=0x7f9294062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9294060860 0x7f9294062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f92b405e820 tx=0x7f92a4009040 comp rx=0 tx=0).stop 2024-08-18T13:40:13.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92b4068820 msgr2=0x7f92b414a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92b4068820 0x7f92b414a0a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f929c00ec80 tx=0x7f929c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 shutdown_connections 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9294060860 0x7f9294062d10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92b40c8d40 0x7f92b40c2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92b40c0b00 0x7f92b414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 --2- 172.21.15.118:0/2054386878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92b4068820 0x7f92b414a0a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 >> 172.21.15.118:0/2054386878 conn(0x7f92b40cc630 msgr2=0x7f92b40c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 shutdown_connections 2024-08-18T13:40:13.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.826+0000 7f92b8956700 1 -- 172.21.15.118:0/2054386878 wait complete. 2024-08-18T13:40:13.843 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-18T13:40:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfef4f7700 1 Processor -- start 2024-08-18T13:40:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfef4f7700 1 -- start start 2024-08-18T13:40:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfef4f7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbfe80c0cf0 0x7fbfe8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfef4f7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbfe80cc430 0x7fbfe814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfef4f7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe80698b0 con 0x7fbfe80c10c0 2024-08-18T13:40:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfef4f7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe8156250 con 0x7fbfe80cc430 2024-08-18T13:40:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfef4f7700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe806a450 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfeda94700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fbfe80cc430 0x7fbfe814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfeca92700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbfe80c10c0 0x7fbfe8069d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53892/0 (socket says 172.21.15.118:53892) 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfeca92700 1 -- 172.21.15.118:0/328941384 learned_addr learned my addr 172.21.15.118:0/328941384 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.954+0000 7fbfed293700 1 --2- 172.21.15.118:0/328941384 >> v2:172.21.15.178:3300/0 conn(0x7fbfe80c0cf0 0x7fbfe8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfdffff700 1 -- 172.21.15.118:0/328941384 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687445432 0 0) 0x7fbfe80698b0 con 0x7fbfe80c10c0 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfdffff700 1 -- 172.21.15.118:0/328941384 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd4003570 con 0x7fbfe80c10c0 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfeda94700 1 -- 172.21.15.118:0/328941384 >> v2:172.21.15.178:3300/0 conn(0x7fbfe80c0cf0 msgr2=0x7fbfe8155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfeda94700 1 --2- 172.21.15.118:0/328941384 >> v2:172.21.15.178:3300/0 conn(0x7fbfe80c0cf0 0x7fbfe8155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfeda94700 1 -- 172.21.15.118:0/328941384 >> v1:172.21.15.118:6789/0 conn(0x7fbfe80c10c0 legacy=0x7fbfe8069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfeda94700 1 -- 172.21.15.118:0/328941384 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80ccb70 con 0x7fbfe80cc430 2024-08-18T13:40:13.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfeda94700 1 --2- 172.21.15.118:0/328941384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80cc430 0x7fbfe814efc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80cbc80 tx=0x7fbfe0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2280e634ef100f5d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfdffff700 1 -- 172.21.15.118:0/328941384 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000e070 con 0x7fbfe80cc430 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfdffff700 1 -- 172.21.15.118:0/328941384 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000bb40 con 0x7fbfe80cc430 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfdffff700 1 -- 172.21.15.118:0/328941384 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0012430 con 0x7fbfe80cc430 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 -- 172.21.15.118:0/328941384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80cc430 msgr2=0x7fbfe814efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/328941384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80cc430 0x7fbfe814efc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80cbc80 tx=0x7fbfe0009780 comp rx=0 tx=0).stop 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 -- 172.21.15.118:0/328941384 shutdown_connections 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/328941384 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80cc430 0x7fbfe814efc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/328941384 >> v2:172.21.15.178:3300/0 conn(0x7fbfe80c0cf0 0x7fbfe8155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 -- 172.21.15.118:0/328941384 >> 172.21.15.118:0/328941384 conn(0x7fbfe8067df0 msgr2=0x7fbfe80cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 -- 172.21.15.118:0/328941384 shutdown_connections 2024-08-18T13:40:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.958+0000 7fbfef4f7700 1 -- 172.21.15.118:0/328941384 wait complete. 2024-08-18T13:40:13.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 Processor -- start 2024-08-18T13:40:13.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 -- start start 2024-08-18T13:40:13.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80c0cf0 0x7fbfe81555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfe80c10c0 0x7fbfe80b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfe80cc430 0x7fbfe8167f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80c31a0 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80b0890 con 0x7fbfe80c10c0 2024-08-18T13:40:13.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfef4f7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80c36d0 con 0x7fbfe80cc430 2024-08-18T13:40:13.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfed293700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80c0cf0 0x7fbfe81555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfed293700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80c0cf0 0x7fbfe81555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51926/0 (socket says 172.21.15.118:51926) 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfed293700 1 -- 172.21.15.118:0/3644055875 learned_addr learned my addr 172.21.15.118:0/3644055875 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfeda94700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfe80cc430 0x7fbfe8167f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.962+0000 7fbfeca92700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfe80c10c0 0x7fbfe80b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfed293700 1 -- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfe80cc430 msgr2=0x7fbfe8167f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfed293700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfe80cc430 0x7fbfe8167f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfed293700 1 -- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfe80c10c0 msgr2=0x7fbfe80b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfed293700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfe80c10c0 0x7fbfe80b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfed293700 1 -- 172.21.15.118:0/3644055875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0009a20 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfed293700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80c0cf0 0x7fbfe81555e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800ecb0 tx=0x7fbfd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfdd7fa700 1 -- 172.21.15.118:0/3644055875 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800c8e0 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfdd7fa700 1 -- 172.21.15.118:0/3644055875 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd800ca40 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe8168a80 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfdd7fa700 1 -- 172.21.15.118:0/3644055875 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800cde0 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe80b0520 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe80642f0 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfdd7fa700 1 -- 172.21.15.118:0/3644055875 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbfd8013680 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.966+0000 7fbfdd7fa700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbfcc060860 0x7fbfcc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:13.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.970+0000 7fbfdd7fa700 1 -- 172.21.15.118:0/3644055875 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fbfd807d190 con 0x7fbfe80c0cf0 2024-08-18T13:40:13.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.970+0000 7fbfeca92700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbfcc060860 0x7fbfcc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:13.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.970+0000 7fbfeca92700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbfcc060860 0x7fbfcc062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80ca030 tx=0x7fbfe4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:13.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:13.970+0000 7fbfdd7fa700 1 -- 172.21.15.118:0/3644055875 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbfd804d2f0 con 0x7fbfe80c0cf0 2024-08-18T13:40:14.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.134+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbfe81553c0 con 0x7fbfe80c0cf0 2024-08-18T13:40:14.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.134+0000 7fbfdd7fa700 1 -- 172.21.15.118:0/3644055875 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbfd8050910 con 0x7fbfe80c0cf0 2024-08-18T13:40:14.140 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:14.142 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbfcc060860 msgr2=0x7fbfcc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.142 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbfcc060860 0x7fbfcc062d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80ca030 tx=0x7fbfe4009040 comp rx=0 tx=0).stop 2024-08-18T13:40:14.142 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80c0cf0 msgr2=0x7fbfe81555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80c0cf0 0x7fbfe81555e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800ecb0 tx=0x7fbfd80091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 shutdown_connections 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbfcc060860 0x7fbfcc062d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfe80cc430 0x7fbfe8167f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfe80c10c0 0x7fbfe80b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 --2- 172.21.15.118:0/3644055875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbfe80c0cf0 0x7fbfe81555e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 >> 172.21.15.118:0/3644055875 conn(0x7fbfe8067df0 msgr2=0x7fbfe8159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 shutdown_connections 2024-08-18T13:40:14.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.138+0000 7fbfef4f7700 1 -- 172.21.15.118:0/3644055875 wait complete. 2024-08-18T13:40:14.154 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-08-18T13:40:14.154 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:40:14.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 Processor -- start 2024-08-18T13:40:14.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- start start 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fed300be340 0x7fed300be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed300c3020 con 0x7fed3015a310 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed300c38c0 con 0x7fed300c3d50 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3006e3b0 con 0x7fed300be340 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35c6a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fed300be340 0x7fed300be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35c6a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fed300be340 0x7fed300be710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51928/0 (socket says 172.21.15.118:51928) 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35c6a700 1 -- 172.21.15.118:0/1957504108 learned_addr learned my addr 172.21.15.118:0/1957504108 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:14.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35469700 1 --1- 172.21.15.118:0/1957504108 >> v1:172.21.15.118:6789/0 conn(0x7fed3015a310 0x7fed3006db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53904/0 (socket says 172.21.15.118:53904) 2024-08-18T13:40:14.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35c6a700 1 -- 172.21.15.118:0/1957504108 >> v1:172.21.15.178:6789/0 conn(0x7fed300c3d50 legacy=0x7fed3006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35c6a700 1 -- 172.21.15.118:0/1957504108 >> v1:172.21.15.118:6789/0 conn(0x7fed3015a310 legacy=0x7fed3006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35c6a700 1 -- 172.21.15.118:0/1957504108 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300673d0 con 0x7fed300be340 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed35c6a700 1 --2- 172.21.15.118:0/1957504108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed300be710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed28009f20 tx=0x7fed28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e7db61f04b5715 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed34c68700 1 -- 172.21.15.118:0/1957504108 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800e070 con 0x7fed300be340 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed34c68700 1 -- 172.21.15.118:0/1957504108 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2800ba40 con 0x7fed300be340 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed34c68700 1 -- 172.21.15.118:0/1957504108 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280092e0 con 0x7fed300be340 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- 172.21.15.118:0/1957504108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 msgr2=0x7fed300be710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 --2- 172.21.15.118:0/1957504108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed300be710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fed28009f20 tx=0x7fed28009840 comp rx=0 tx=0).stop 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- 172.21.15.118:0/1957504108 shutdown_connections 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 --2- 172.21.15.118:0/1957504108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed300be710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- 172.21.15.118:0/1957504108 >> 172.21.15.118:0/1957504108 conn(0x7fed300c7c40 msgr2=0x7fed300c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:14.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- 172.21.15.118:0/1957504108 shutdown_connections 2024-08-18T13:40:14.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.246+0000 7fed37ece700 1 -- 172.21.15.118:0/1957504108 wait complete. 2024-08-18T13:40:14.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 Processor -- start 2024-08-18T13:40:14.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 -- start start 2024-08-18T13:40:14.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed30154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed300c3d50 0x7fed30158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed3015a310 0x7fed30150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300c78d0 con 0x7fed300be340 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3016a6e0 con 0x7fed3015a310 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3016a9c0 con 0x7fed300c3d50 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed30154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed30154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51936/0 (socket says 172.21.15.118:51936) 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 -- 172.21.15.118:0/4057372481 learned_addr learned my addr 172.21.15.118:0/4057372481 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35469700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed300c3d50 0x7fed30158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed3646b700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed3015a310 0x7fed30150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 -- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed300c3d50 msgr2=0x7fed30158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed300c3d50 0x7fed30158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 -- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed3015a310 msgr2=0x7fed30150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed3015a310 0x7fed30150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 -- 172.21.15.118:0/4057372481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28009ae0 con 0x7fed300be340 2024-08-18T13:40:14.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed35c6a700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed30154570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fed3006d1d0 tx=0x7fed28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed25ffb700 1 -- 172.21.15.118:0/4057372481 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28012950 con 0x7fed300be340 2024-08-18T13:40:14.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed25ffb700 1 -- 172.21.15.118:0/4057372481 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28012ab0 con 0x7fed300be340 2024-08-18T13:40:14.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed301560f0 con 0x7fed300be340 2024-08-18T13:40:14.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed25ffb700 1 -- 172.21.15.118:0/4057372481 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28012e50 con 0x7fed300be340 2024-08-18T13:40:14.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.250+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed301566b0 con 0x7fed300be340 2024-08-18T13:40:14.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.254+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed300642f0 con 0x7fed300be340 2024-08-18T13:40:14.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.254+0000 7fed25ffb700 1 -- 172.21.15.118:0/4057372481 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fed2801d630 con 0x7fed300be340 2024-08-18T13:40:14.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.254+0000 7fed25ffb700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fed14060860 0x7fed14062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.254+0000 7fed35469700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fed14060860 0x7fed14062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.254+0000 7fed25ffb700 1 -- 172.21.15.118:0/4057372481 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7fed2807eaa0 con 0x7fed300be340 2024-08-18T13:40:14.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.258+0000 7fed35469700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fed14060860 0x7fed14062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed300c3b00 tx=0x7fed20007eb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.258+0000 7fed25ffb700 1 -- 172.21.15.118:0/4057372481 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fed2804eb60 con 0x7fed300be340 2024-08-18T13:40:14.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.458+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed30069a90 con 0x7fed300be340 2024-08-18T13:40:14.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed25ffb700 1 -- 172.21.15.118:0/4057372481 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7fed28052180 con 0x7fed300be340 2024-08-18T13:40:14.466 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:14.466 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:40:14.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fed14060860 msgr2=0x7fed14062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fed14060860 0x7fed14062d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed300c3b00 tx=0x7fed20007eb0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 msgr2=0x7fed30154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed30154570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fed3006d1d0 tx=0x7fed28017000 comp rx=0 tx=0).stop 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 shutdown_connections 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fed14060860 0x7fed14062d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed3015a310 0x7fed30150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fed300c3d50 0x7fed30158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 --2- 172.21.15.118:0/4057372481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed300be340 0x7fed30154570 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 >> 172.21.15.118:0/4057372481 conn(0x7fed300c7c40 msgr2=0x7fed30152100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 shutdown_connections 2024-08-18T13:40:14.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.462+0000 7fed37ece700 1 -- 172.21.15.118:0/4057372481 wait complete. 2024-08-18T13:40:14.470 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:40:14.480 INFO:tasks.mds_thrash:Ready to start thrashing 2024-08-18T13:40:14.481 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-08-18T13:40:14.481 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:40:14.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 Processor -- start 2024-08-18T13:40:14.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- start start 2024-08-18T13:40:14.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6cf0153690 0x7f6cf00673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf006ee60 con 0x7f6cf006f2f0 2024-08-18T13:40:14.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf006f730 con 0x7f6cf00c8200 2024-08-18T13:40:14.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf015a640 con 0x7f6cf0153690 2024-08-18T13:40:14.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceeffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6cf0153690 0x7f6cf00673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceeffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f6cf0153690 0x7f6cf00673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51940/0 (socket says 172.21.15.118:51940) 2024-08-18T13:40:14.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceeffd700 1 -- 172.21.15.118:0/713801419 learned_addr learned my addr 172.21.15.118:0/713801419 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:14.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6cef7fe700 1 --1- 172.21.15.118:0/713801419 >> v1:172.21.15.118:6789/0 conn(0x7f6cf006f2f0 0x7f6cf00c29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53916/0 (socket says 172.21.15.118:53916) 2024-08-18T13:40:14.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceeffd700 1 -- 172.21.15.118:0/713801419 >> v1:172.21.15.178:6789/0 conn(0x7f6cf00c8200 legacy=0x7f6cf00c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:14.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceeffd700 1 -- 172.21.15.118:0/713801419 >> v1:172.21.15.118:6789/0 conn(0x7f6cf006f2f0 legacy=0x7f6cf00c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceeffd700 1 -- 172.21.15.118:0/713801419 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0153d10 con 0x7f6cf0153690 2024-08-18T13:40:14.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6cedffb700 1 -- 172.21.15.118:0/713801419 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572142855 0 0) 0x7f6cf006ee60 con 0x7f6cf006f2f0 2024-08-18T13:40:14.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceeffd700 1 --2- 172.21.15.118:0/713801419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf00673d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40099b0 tx=0x7f6ce4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf9b078c57d6af3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6cedffb700 1 -- 172.21.15.118:0/713801419 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400b860 con 0x7f6cf0153690 2024-08-18T13:40:14.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6cedffb700 1 -- 172.21.15.118:0/713801419 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce400b9c0 con 0x7f6cf0153690 2024-08-18T13:40:14.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6cedffb700 1 -- 172.21.15.118:0/713801419 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400bd60 con 0x7f6cf0153690 2024-08-18T13:40:14.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- 172.21.15.118:0/713801419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 msgr2=0x7f6cf00673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 --2- 172.21.15.118:0/713801419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf00673d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40099b0 tx=0x7f6ce4009250 comp rx=0 tx=0).stop 2024-08-18T13:40:14.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- 172.21.15.118:0/713801419 shutdown_connections 2024-08-18T13:40:14.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 --2- 172.21.15.118:0/713801419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf00673d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- 172.21.15.118:0/713801419 >> 172.21.15.118:0/713801419 conn(0x7f6cf006c020 msgr2=0x7f6cf00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:14.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- 172.21.15.118:0/713801419 shutdown_connections 2024-08-18T13:40:14.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.586+0000 7f6ceffff700 1 -- 172.21.15.118:0/713801419 wait complete. 2024-08-18T13:40:14.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 Processor -- start 2024-08-18T13:40:14.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 -- start start 2024-08-18T13:40:14.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6cf006f2f0 0x7f6cf014eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf00c8200 0x7f6cf0158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf0150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0155a10 con 0x7f6cf0153690 2024-08-18T13:40:14.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00b44e0 con 0x7f6cf00c8200 2024-08-18T13:40:14.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0158390 con 0x7f6cf006f2f0 2024-08-18T13:40:14.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf0150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf0150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51950/0 (socket says 172.21.15.118:51950) 2024-08-18T13:40:14.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 -- 172.21.15.118:0/150340432 learned_addr learned my addr 172.21.15.118:0/150340432 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:14.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6ceeffd700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6cf006f2f0 0x7f6cf014eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cee7fc700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf00c8200 0x7f6cf0158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 -- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6cf006f2f0 msgr2=0x7f6cf014eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6cf006f2f0 0x7f6cf014eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 -- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf00c8200 msgr2=0x7f6cf0158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf00c8200 0x7f6cf0158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 -- 172.21.15.118:0/150340432 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40094f0 con 0x7f6cf0153690 2024-08-18T13:40:14.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.590+0000 7f6cef7fe700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf0150c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0001eb0 tx=0x7f6ce0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6cd6ffd700 1 -- 172.21.15.118:0/150340432 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0005a70 con 0x7f6cf0153690 2024-08-18T13:40:14.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6cd6ffd700 1 -- 172.21.15.118:0/150340432 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0005bd0 con 0x7f6cf0153690 2024-08-18T13:40:14.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf0158670 con 0x7f6cf0153690 2024-08-18T13:40:14.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6cd6ffd700 1 -- 172.21.15.118:0/150340432 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00070d0 con 0x7f6cf0153690 2024-08-18T13:40:14.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf0169e00 con 0x7f6cf0153690 2024-08-18T13:40:14.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf0158800 con 0x7f6cf0153690 2024-08-18T13:40:14.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6cd6ffd700 1 -- 172.21.15.118:0/150340432 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6ce0007440 con 0x7f6cf0153690 2024-08-18T13:40:14.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6cd6ffd700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6cdc060860 0x7f6cdc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6cd6ffd700 1 -- 172.21.15.118:0/150340432 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f6ce007c260 con 0x7f6cf0153690 2024-08-18T13:40:14.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6ceeffd700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6cdc060860 0x7f6cdc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.594+0000 7f6ceeffd700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6cdc060860 0x7f6cdc062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40099b0 tx=0x7f6ce4017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.602+0000 7f6cd6ffd700 1 -- 172.21.15.118:0/150340432 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6ce004c3c0 con 0x7f6cf0153690 2024-08-18T13:40:14.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.762+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cf00713a0 con 0x7f6cdc060860 2024-08-18T13:40:14.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.774+0000 7f6cd6ffd700 1 -- 172.21.15.118:0/150340432 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345233 (secure 0 0 0) 0x7f6cf00713a0 con 0x7f6cdc060860 2024-08-18T13:40:14.783 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6cdc060860 msgr2=0x7f6cdc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6cdc060860 0x7f6cdc062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40099b0 tx=0x7f6ce4017000 comp rx=0 tx=0).stop 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 msgr2=0x7f6cf0150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf0150c90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0001eb0 tx=0x7f6ce0007d10 comp rx=0 tx=0).stop 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 shutdown_connections 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6cdc060860 0x7f6cdc062d10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cf0153690 0x7f6cf0150c90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf00c8200 0x7f6cf0158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 --2- 172.21.15.118:0/150340432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6cf006f2f0 0x7f6cf014eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 >> 172.21.15.118:0/150340432 conn(0x7f6cf006c020 msgr2=0x7f6cf00c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:14.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 shutdown_connections 2024-08-18T13:40:14.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.778+0000 7f6ceffff700 1 -- 172.21.15.118:0/150340432 wait complete. 2024-08-18T13:40:14.788 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:40:14.797 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-08-18T13:40:13.210835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595361,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":209,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415368,"kb_used_data":3240,"kb_used_omap":13,"kb_used_meta":2412018,"kb_avail":752559352,"statfs":{"total":773094113280,"available":770620776448,"internally_reserved":0,"allocated":3317760,"data_stored":2007226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2469907136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":445,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005690"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433629+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:39:40.433629+0000","last_peered":"2024-08-18T13:39:40.433629+0000","last_clean":"2024-08-18T13:39:40.433629+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:39:40.433629+0000","last_undegraded":"2024-08-18T13:39:40.433629+0000","last_fullsized":"2024-08-18T13:39:40.433629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422872+0000","last_change":"2024-08-18T13:39:35.405966+0000","last_active":"2024-08-18T13:39:39.422872+0000","last_peered":"2024-08-18T13:39:39.422872+0000","last_clean":"2024-08-18T13:39:39.422872+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:39:39.422872+0000","last_undegraded":"2024-08-18T13:39:39.422872+0000","last_fullsized":"2024-08-18T13:39:39.422872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454715+0000","last_change":"2024-08-18T13:39:39.453674+0000","last_active":"2024-08-18T13:39:39.454715+0000","last_peered":"2024-08-18T13:39:39.454715+0000","last_clean":"2024-08-18T13:39:39.454715+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:39:39.454715+0000","last_undegraded":"2024-08-18T13:39:39.454715+0000","last_fullsized":"2024-08-18T13:39:39.454715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434410+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:39:40.434410+0000","last_peered":"2024-08-18T13:39:40.434410+0000","last_clean":"2024-08-18T13:39:40.434410+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:39:40.434410+0000","last_undegraded":"2024-08-18T13:39:40.434410+0000","last_fullsized":"2024-08-18T13:39:40.434410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432862+0000","last_change":"2024-08-18T13:39:35.408917+0000","last_active":"2024-08-18T13:39:39.432862+0000","last_peered":"2024-08-18T13:39:39.432862+0000","last_clean":"2024-08-18T13:39:39.432862+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:39:39.432862+0000","last_undegraded":"2024-08-18T13:39:39.432862+0000","last_fullsized":"2024-08-18T13:39:39.432862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447766+0000","last_change":"2024-08-18T13:39:39.445388+0000","last_active":"2024-08-18T13:39:39.447766+0000","last_peered":"2024-08-18T13:39:39.447766+0000","last_clean":"2024-08-18T13:39:39.447766+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:39:39.447766+0000","last_undegraded":"2024-08-18T13:39:39.447766+0000","last_fullsized":"2024-08-18T13:39:39.447766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430049+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:39:40.430049+0000","last_peered":"2024-08-18T13:39:40.430049+0000","last_clean":"2024-08-18T13:39:40.430049+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:39:40.430049+0000","last_undegraded":"2024-08-18T13:39:40.430049+0000","last_fullsized":"2024-08-18T13:39:40.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422649+0000","last_change":"2024-08-18T13:39:35.399188+0000","last_active":"2024-08-18T13:39:39.422649+0000","last_peered":"2024-08-18T13:39:39.422649+0000","last_clean":"2024-08-18T13:39:39.422649+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:39:39.422649+0000","last_undegraded":"2024-08-18T13:39:39.422649+0000","last_fullsized":"2024-08-18T13:39:39.422649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447285+0000","last_change":"2024-08-18T13:39:39.440922+0000","last_active":"2024-08-18T13:39:39.447285+0000","last_peered":"2024-08-18T13:39:39.447285+0000","last_clean":"2024-08-18T13:39:39.447285+0000","last_became_active":"2024-08-18T13:39:39.440426+0000","last_became_peered":"2024-08-18T13:39:39.440426+0000","last_unstale":"2024-08-18T13:39:39.447285+0000","last_undegraded":"2024-08-18T13:39:39.447285+0000","last_fullsized":"2024-08-18T13:39:39.447285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:23:00.376805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.428764+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:39:40.428764+0000","last_peered":"2024-08-18T13:39:40.428764+0000","last_clean":"2024-08-18T13:39:40.428764+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:39:40.428764+0000","last_undegraded":"2024-08-18T13:39:40.428764+0000","last_fullsized":"2024-08-18T13:39:40.428764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423184+0000","last_change":"2024-08-18T13:39:35.400038+0000","last_active":"2024-08-18T13:39:39.423184+0000","last_peered":"2024-08-18T13:39:39.423184+0000","last_clean":"2024-08-18T13:39:39.423184+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:39:39.423184+0000","last_undegraded":"2024-08-18T13:39:39.423184+0000","last_fullsized":"2024-08-18T13:39:39.423184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437377+0000","last_change":"2024-08-18T13:39:39.436221+0000","last_active":"2024-08-18T13:39:39.437377+0000","last_peered":"2024-08-18T13:39:39.437377+0000","last_clean":"2024-08-18T13:39:39.437377+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:39:39.437377+0000","last_undegraded":"2024-08-18T13:39:39.437377+0000","last_fullsized":"2024-08-18T13:39:39.437377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428676+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.428676+0000","last_peered":"2024-08-18T13:39:39.428676+0000","last_clean":"2024-08-18T13:39:39.428676+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:39:39.428676+0000","last_undegraded":"2024-08-18T13:39:39.428676+0000","last_fullsized":"2024-08-18T13:39:39.428676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430090+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:39:40.430090+0000","last_peered":"2024-08-18T13:39:40.430090+0000","last_clean":"2024-08-18T13:39:40.430090+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:39:40.430090+0000","last_undegraded":"2024-08-18T13:39:40.430090+0000","last_fullsized":"2024-08-18T13:39:40.430090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.443080+0000","last_change":"2024-08-18T13:39:39.439633+0000","last_active":"2024-08-18T13:39:39.443080+0000","last_peered":"2024-08-18T13:39:39.443080+0000","last_clean":"2024-08-18T13:39:39.443080+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:39:39.443080+0000","last_undegraded":"2024-08-18T13:39:39.443080+0000","last_fullsized":"2024-08-18T13:39:39.443080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423607+0000","last_change":"2024-08-18T13:39:35.407947+0000","last_active":"2024-08-18T13:39:39.423607+0000","last_peered":"2024-08-18T13:39:39.423607+0000","last_clean":"2024-08-18T13:39:39.423607+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:39:39.423607+0000","last_undegraded":"2024-08-18T13:39:39.423607+0000","last_fullsized":"2024-08-18T13:39:39.423607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429156+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:39:39.429156+0000","last_peered":"2024-08-18T13:39:39.429156+0000","last_clean":"2024-08-18T13:39:39.429156+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:39:39.429156+0000","last_undegraded":"2024-08-18T13:39:39.429156+0000","last_fullsized":"2024-08-18T13:39:39.429156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442892+0000","last_change":"2024-08-18T13:39:39.439913+0000","last_active":"2024-08-18T13:39:39.442892+0000","last_peered":"2024-08-18T13:39:39.442892+0000","last_clean":"2024-08-18T13:39:39.442892+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:39:39.442892+0000","last_undegraded":"2024-08-18T13:39:39.442892+0000","last_fullsized":"2024-08-18T13:39:39.442892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430332+0000","last_change":"2024-08-18T13:39:35.403254+0000","last_active":"2024-08-18T13:39:39.430332+0000","last_peered":"2024-08-18T13:39:39.430332+0000","last_clean":"2024-08-18T13:39:39.430332+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:39:39.430332+0000","last_undegraded":"2024-08-18T13:39:39.430332+0000","last_fullsized":"2024-08-18T13:39:39.430332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422689+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:39:39.422689+0000","last_peered":"2024-08-18T13:39:39.422689+0000","last_clean":"2024-08-18T13:39:39.422689+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:39:39.422689+0000","last_undegraded":"2024-08-18T13:39:39.422689+0000","last_fullsized":"2024-08-18T13:39:39.422689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457822+0000","last_change":"2024-08-18T13:39:39.457250+0000","last_active":"2024-08-18T13:39:39.457822+0000","last_peered":"2024-08-18T13:39:39.457822+0000","last_clean":"2024-08-18T13:39:39.457822+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:39:39.457822+0000","last_undegraded":"2024-08-18T13:39:39.457822+0000","last_fullsized":"2024-08-18T13:39:39.457822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425047+0000","last_change":"2024-08-18T13:39:35.401612+0000","last_active":"2024-08-18T13:39:39.425047+0000","last_peered":"2024-08-18T13:39:39.425047+0000","last_clean":"2024-08-18T13:39:39.425047+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:39:39.425047+0000","last_undegraded":"2024-08-18T13:39:39.425047+0000","last_fullsized":"2024-08-18T13:39:39.425047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450234+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:39:39.450234+0000","last_peered":"2024-08-18T13:39:39.450234+0000","last_clean":"2024-08-18T13:39:39.450234+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:39:39.450234+0000","last_undegraded":"2024-08-18T13:39:39.450234+0000","last_fullsized":"2024-08-18T13:39:39.450234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447688+0000","last_change":"2024-08-18T13:39:39.443503+0000","last_active":"2024-08-18T13:39:39.447688+0000","last_peered":"2024-08-18T13:39:39.447688+0000","last_clean":"2024-08-18T13:39:39.447688+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:39:39.447688+0000","last_undegraded":"2024-08-18T13:39:39.447688+0000","last_fullsized":"2024-08-18T13:39:39.447688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432976+0000","last_change":"2024-08-18T13:39:35.407907+0000","last_active":"2024-08-18T13:39:39.432976+0000","last_peered":"2024-08-18T13:39:39.432976+0000","last_clean":"2024-08-18T13:39:39.432976+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:39:39.432976+0000","last_undegraded":"2024-08-18T13:39:39.432976+0000","last_fullsized":"2024-08-18T13:39:39.432976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429181+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:39:40.429181+0000","last_peered":"2024-08-18T13:39:40.429181+0000","last_clean":"2024-08-18T13:39:40.429181+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:39:40.429181+0000","last_undegraded":"2024-08-18T13:39:40.429181+0000","last_fullsized":"2024-08-18T13:39:40.429181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454767+0000","last_change":"2024-08-18T13:39:39.453138+0000","last_active":"2024-08-18T13:39:39.454767+0000","last_peered":"2024-08-18T13:39:39.454767+0000","last_clean":"2024-08-18T13:39:39.454767+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:39:39.454767+0000","last_undegraded":"2024-08-18T13:39:39.454767+0000","last_fullsized":"2024-08-18T13:39:39.454767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424552+0000","last_change":"2024-08-18T13:39:35.404453+0000","last_active":"2024-08-18T13:39:39.424552+0000","last_peered":"2024-08-18T13:39:39.424552+0000","last_clean":"2024-08-18T13:39:39.424552+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:39:39.424552+0000","last_undegraded":"2024-08-18T13:39:39.424552+0000","last_fullsized":"2024-08-18T13:39:39.424552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:41.051281+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:39:41.051281+0000","last_peered":"2024-08-18T13:39:41.051281+0000","last_clean":"2024-08-18T13:39:41.051281+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:39:41.051281+0000","last_undegraded":"2024-08-18T13:39:41.051281+0000","last_fullsized":"2024-08-18T13:39:41.051281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450645+0000","last_change":"2024-08-18T13:39:39.449851+0000","last_active":"2024-08-18T13:39:39.450645+0000","last_peered":"2024-08-18T13:39:39.450645+0000","last_clean":"2024-08-18T13:39:39.450645+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:39:39.450645+0000","last_undegraded":"2024-08-18T13:39:39.450645+0000","last_fullsized":"2024-08-18T13:39:39.450645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424459+0000","last_change":"2024-08-18T13:39:35.400604+0000","last_active":"2024-08-18T13:39:39.424459+0000","last_peered":"2024-08-18T13:39:39.424459+0000","last_clean":"2024-08-18T13:39:39.424459+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:39:39.424459+0000","last_undegraded":"2024-08-18T13:39:39.424459+0000","last_fullsized":"2024-08-18T13:39:39.424459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450871+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:39:39.450871+0000","last_peered":"2024-08-18T13:39:39.450871+0000","last_clean":"2024-08-18T13:39:39.450871+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:39:39.450871+0000","last_undegraded":"2024-08-18T13:39:39.450871+0000","last_fullsized":"2024-08-18T13:39:39.450871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454096+0000","last_change":"2024-08-18T13:39:39.453599+0000","last_active":"2024-08-18T13:39:39.454096+0000","last_peered":"2024-08-18T13:39:39.454096+0000","last_clean":"2024-08-18T13:39:39.454096+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:39:39.454096+0000","last_undegraded":"2024-08-18T13:39:39.454096+0000","last_fullsized":"2024-08-18T13:39:39.454096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423160+0000","last_change":"2024-08-18T13:39:35.410167+0000","last_active":"2024-08-18T13:39:39.423160+0000","last_peered":"2024-08-18T13:39:39.423160+0000","last_clean":"2024-08-18T13:39:39.423160+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:39:39.423160+0000","last_undegraded":"2024-08-18T13:39:39.423160+0000","last_fullsized":"2024-08-18T13:39:39.423160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424551+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:39:39.424551+0000","last_peered":"2024-08-18T13:39:39.424551+0000","last_clean":"2024-08-18T13:39:39.424551+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:39:39.424551+0000","last_undegraded":"2024-08-18T13:39:39.424551+0000","last_fullsized":"2024-08-18T13:39:39.424551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444517+0000","last_change":"2024-08-18T13:39:39.439948+0000","last_active":"2024-08-18T13:39:39.444517+0000","last_peered":"2024-08-18T13:39:39.444517+0000","last_clean":"2024-08-18T13:39:39.444517+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:39:39.444517+0000","last_undegraded":"2024-08-18T13:39:39.444517+0000","last_fullsized":"2024-08-18T13:39:39.444517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422894+0000","last_change":"2024-08-18T13:39:35.407829+0000","last_active":"2024-08-18T13:39:39.422894+0000","last_peered":"2024-08-18T13:39:39.422894+0000","last_clean":"2024-08-18T13:39:39.422894+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:39:39.422894+0000","last_undegraded":"2024-08-18T13:39:39.422894+0000","last_fullsized":"2024-08-18T13:39:39.422894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417877+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:39:39.417877+0000","last_peered":"2024-08-18T13:39:39.417877+0000","last_clean":"2024-08-18T13:39:39.417877+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:39:39.417877+0000","last_undegraded":"2024-08-18T13:39:39.417877+0000","last_fullsized":"2024-08-18T13:39:39.417877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445479+0000","last_change":"2024-08-18T13:39:39.442372+0000","last_active":"2024-08-18T13:39:39.445479+0000","last_peered":"2024-08-18T13:39:39.445479+0000","last_clean":"2024-08-18T13:39:39.445479+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:39:39.445479+0000","last_undegraded":"2024-08-18T13:39:39.445479+0000","last_fullsized":"2024-08-18T13:39:39.445479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421862+0000","last_change":"2024-08-18T13:39:35.400384+0000","last_active":"2024-08-18T13:39:39.421862+0000","last_peered":"2024-08-18T13:39:39.421862+0000","last_clean":"2024-08-18T13:39:39.421862+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:39:39.421862+0000","last_undegraded":"2024-08-18T13:39:39.421862+0000","last_fullsized":"2024-08-18T13:39:39.421862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415832+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:39:39.415832+0000","last_peered":"2024-08-18T13:39:39.415832+0000","last_clean":"2024-08-18T13:39:39.415832+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:39:39.415832+0000","last_undegraded":"2024-08-18T13:39:39.415832+0000","last_fullsized":"2024-08-18T13:39:39.415832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446859+0000","last_change":"2024-08-18T13:39:39.446401+0000","last_active":"2024-08-18T13:39:39.446859+0000","last_peered":"2024-08-18T13:39:39.446859+0000","last_clean":"2024-08-18T13:39:39.446859+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:39:39.446859+0000","last_undegraded":"2024-08-18T13:39:39.446859+0000","last_fullsized":"2024-08-18T13:39:39.446859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424307+0000","last_change":"2024-08-18T13:39:35.410127+0000","last_active":"2024-08-18T13:39:39.424307+0000","last_peered":"2024-08-18T13:39:39.424307+0000","last_clean":"2024-08-18T13:39:39.424307+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:39:39.424307+0000","last_undegraded":"2024-08-18T13:39:39.424307+0000","last_fullsized":"2024-08-18T13:39:39.424307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450338+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:39:39.450338+0000","last_peered":"2024-08-18T13:39:39.450338+0000","last_clean":"2024-08-18T13:39:39.450338+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:39:39.450338+0000","last_undegraded":"2024-08-18T13:39:39.450338+0000","last_fullsized":"2024-08-18T13:39:39.450338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454861+0000","last_change":"2024-08-18T13:39:39.453901+0000","last_active":"2024-08-18T13:39:39.454861+0000","last_peered":"2024-08-18T13:39:39.454861+0000","last_clean":"2024-08-18T13:39:39.454861+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:39:39.454861+0000","last_undegraded":"2024-08-18T13:39:39.454861+0000","last_fullsized":"2024-08-18T13:39:39.454861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430338+0000","last_change":"2024-08-18T13:39:35.412156+0000","last_active":"2024-08-18T13:39:39.430338+0000","last_peered":"2024-08-18T13:39:39.430338+0000","last_clean":"2024-08-18T13:39:39.430338+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:39:39.430338+0000","last_undegraded":"2024-08-18T13:39:39.430338+0000","last_fullsized":"2024-08-18T13:39:39.430338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417339+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:39:39.417339+0000","last_peered":"2024-08-18T13:39:39.417339+0000","last_clean":"2024-08-18T13:39:39.417339+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:39:39.417339+0000","last_undegraded":"2024-08-18T13:39:39.417339+0000","last_fullsized":"2024-08-18T13:39:39.417339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450727+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:39:39.450727+0000","last_peered":"2024-08-18T13:39:39.450727+0000","last_clean":"2024-08-18T13:39:39.450727+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:39:39.450727+0000","last_undegraded":"2024-08-18T13:39:39.450727+0000","last_fullsized":"2024-08-18T13:39:39.450727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454332+0000","last_change":"2024-08-18T13:39:39.453360+0000","last_active":"2024-08-18T13:39:39.454332+0000","last_peered":"2024-08-18T13:39:39.454332+0000","last_clean":"2024-08-18T13:39:39.454332+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:39:39.454332+0000","last_undegraded":"2024-08-18T13:39:39.454332+0000","last_fullsized":"2024-08-18T13:39:39.454332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450556+0000","last_change":"2024-08-18T13:39:35.421765+0000","last_active":"2024-08-18T13:39:39.450556+0000","last_peered":"2024-08-18T13:39:39.450556+0000","last_clean":"2024-08-18T13:39:39.450556+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:39:39.450556+0000","last_undegraded":"2024-08-18T13:39:39.450556+0000","last_fullsized":"2024-08-18T13:39:39.450556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416404+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:39:39.416404+0000","last_peered":"2024-08-18T13:39:39.416404+0000","last_clean":"2024-08-18T13:39:39.416404+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:39:39.416404+0000","last_undegraded":"2024-08-18T13:39:39.416404+0000","last_fullsized":"2024-08-18T13:39:39.416404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450222+0000","last_change":"2024-08-18T13:39:39.449653+0000","last_active":"2024-08-18T13:39:39.450222+0000","last_peered":"2024-08-18T13:39:39.450222+0000","last_clean":"2024-08-18T13:39:39.450222+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:39:39.450222+0000","last_undegraded":"2024-08-18T13:39:39.450222+0000","last_fullsized":"2024-08-18T13:39:39.450222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422710+0000","last_change":"2024-08-18T13:39:35.407439+0000","last_active":"2024-08-18T13:39:39.422710+0000","last_peered":"2024-08-18T13:39:39.422710+0000","last_clean":"2024-08-18T13:39:39.422710+0000","last_became_active":"2024-08-18T13:39:35.407128+0000","last_became_peered":"2024-08-18T13:39:35.407128+0000","last_unstale":"2024-08-18T13:39:39.422710+0000","last_undegraded":"2024-08-18T13:39:39.422710+0000","last_fullsized":"2024-08-18T13:39:39.422710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431851+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:39:40.431851+0000","last_peered":"2024-08-18T13:39:40.431851+0000","last_clean":"2024-08-18T13:39:40.431851+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:39:40.431851+0000","last_undegraded":"2024-08-18T13:39:40.431851+0000","last_fullsized":"2024-08-18T13:39:40.431851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445844+0000","last_change":"2024-08-18T13:39:39.443848+0000","last_active":"2024-08-18T13:39:39.445844+0000","last_peered":"2024-08-18T13:39:39.445844+0000","last_clean":"2024-08-18T13:39:39.445844+0000","last_became_active":"2024-08-18T13:39:39.443456+0000","last_became_peered":"2024-08-18T13:39:39.443456+0000","last_unstale":"2024-08-18T13:39:39.445844+0000","last_undegraded":"2024-08-18T13:39:39.445844+0000","last_fullsized":"2024-08-18T13:39:39.445844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:27.994932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450757+0000","last_change":"2024-08-18T13:39:35.421612+0000","last_active":"2024-08-18T13:39:39.450757+0000","last_peered":"2024-08-18T13:39:39.450757+0000","last_clean":"2024-08-18T13:39:39.450757+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:39:39.450757+0000","last_undegraded":"2024-08-18T13:39:39.450757+0000","last_fullsized":"2024-08-18T13:39:39.450757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433442+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:39:40.433442+0000","last_peered":"2024-08-18T13:39:40.433442+0000","last_clean":"2024-08-18T13:39:40.433442+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:39:40.433442+0000","last_undegraded":"2024-08-18T13:39:40.433442+0000","last_fullsized":"2024-08-18T13:39:40.433442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454815+0000","last_change":"2024-08-18T13:39:39.453520+0000","last_active":"2024-08-18T13:39:39.454815+0000","last_peered":"2024-08-18T13:39:39.454815+0000","last_clean":"2024-08-18T13:39:39.454815+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:39:39.454815+0000","last_undegraded":"2024-08-18T13:39:39.454815+0000","last_fullsized":"2024-08-18T13:39:39.454815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450976+0000","last_change":"2024-08-18T13:39:35.421465+0000","last_active":"2024-08-18T13:39:39.450976+0000","last_peered":"2024-08-18T13:39:39.450976+0000","last_clean":"2024-08-18T13:39:39.450976+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:39:39.450976+0000","last_undegraded":"2024-08-18T13:39:39.450976+0000","last_fullsized":"2024-08-18T13:39:39.450976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429394+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:39:40.429394+0000","last_peered":"2024-08-18T13:39:40.429394+0000","last_clean":"2024-08-18T13:39:40.429394+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:39:40.429394+0000","last_undegraded":"2024-08-18T13:39:40.429394+0000","last_fullsized":"2024-08-18T13:39:40.429394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.448067+0000","last_change":"2024-08-18T13:39:39.446029+0000","last_active":"2024-08-18T13:39:39.448067+0000","last_peered":"2024-08-18T13:39:39.448067+0000","last_clean":"2024-08-18T13:39:39.448067+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:39:39.448067+0000","last_undegraded":"2024-08-18T13:39:39.448067+0000","last_fullsized":"2024-08-18T13:39:39.448067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450468+0000","last_change":"2024-08-18T13:39:35.417094+0000","last_active":"2024-08-18T13:39:39.450468+0000","last_peered":"2024-08-18T13:39:39.450468+0000","last_clean":"2024-08-18T13:39:39.450468+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:39:39.450468+0000","last_undegraded":"2024-08-18T13:39:39.450468+0000","last_fullsized":"2024-08-18T13:39:39.450468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.436721+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:39:40.436721+0000","last_peered":"2024-08-18T13:39:40.436721+0000","last_clean":"2024-08-18T13:39:40.436721+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:39:40.436721+0000","last_undegraded":"2024-08-18T13:39:40.436721+0000","last_fullsized":"2024-08-18T13:39:40.436721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437633+0000","last_change":"2024-08-18T13:39:39.436992+0000","last_active":"2024-08-18T13:39:39.437633+0000","last_peered":"2024-08-18T13:39:39.437633+0000","last_clean":"2024-08-18T13:39:39.437633+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:39:39.437633+0000","last_undegraded":"2024-08-18T13:39:39.437633+0000","last_fullsized":"2024-08-18T13:39:39.437633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427745+0000","last_change":"2024-08-18T13:39:35.411315+0000","last_active":"2024-08-18T13:39:39.427745+0000","last_peered":"2024-08-18T13:39:39.427745+0000","last_clean":"2024-08-18T13:39:39.427745+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:39:39.427745+0000","last_undegraded":"2024-08-18T13:39:39.427745+0000","last_fullsized":"2024-08-18T13:39:39.427745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428272+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:39:39.428272+0000","last_peered":"2024-08-18T13:39:39.428272+0000","last_clean":"2024-08-18T13:39:39.428272+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:39:39.428272+0000","last_undegraded":"2024-08-18T13:39:39.428272+0000","last_fullsized":"2024-08-18T13:39:39.428272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.458299+0000","last_change":"2024-08-18T13:39:39.456955+0000","last_active":"2024-08-18T13:39:39.458299+0000","last_peered":"2024-08-18T13:39:39.458299+0000","last_clean":"2024-08-18T13:39:39.458299+0000","last_became_active":"2024-08-18T13:39:39.456534+0000","last_became_peered":"2024-08-18T13:39:39.456534+0000","last_unstale":"2024-08-18T13:39:39.458299+0000","last_undegraded":"2024-08-18T13:39:39.458299+0000","last_fullsized":"2024-08-18T13:39:39.458299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:26:42.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450840+0000","last_change":"2024-08-18T13:39:35.417348+0000","last_active":"2024-08-18T13:39:39.450840+0000","last_peered":"2024-08-18T13:39:39.450840+0000","last_clean":"2024-08-18T13:39:39.450840+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:39:39.450840+0000","last_undegraded":"2024-08-18T13:39:39.450840+0000","last_fullsized":"2024-08-18T13:39:39.450840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417725+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:39:39.417725+0000","last_peered":"2024-08-18T13:39:39.417725+0000","last_clean":"2024-08-18T13:39:39.417725+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:39:39.417725+0000","last_undegraded":"2024-08-18T13:39:39.417725+0000","last_fullsized":"2024-08-18T13:39:39.417725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454724+0000","last_change":"2024-08-18T13:39:39.453682+0000","last_active":"2024-08-18T13:39:39.454724+0000","last_peered":"2024-08-18T13:39:39.454724+0000","last_clean":"2024-08-18T13:39:39.454724+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:39:39.454724+0000","last_undegraded":"2024-08-18T13:39:39.454724+0000","last_fullsized":"2024-08-18T13:39:39.454724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427706+0000","last_change":"2024-08-18T13:39:35.407127+0000","last_active":"2024-08-18T13:39:39.427706+0000","last_peered":"2024-08-18T13:39:39.427706+0000","last_clean":"2024-08-18T13:39:39.427706+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:39:39.427706+0000","last_undegraded":"2024-08-18T13:39:39.427706+0000","last_fullsized":"2024-08-18T13:39:39.427706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431523+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:39:40.431523+0000","last_peered":"2024-08-18T13:39:40.431523+0000","last_clean":"2024-08-18T13:39:40.431523+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:39:40.431523+0000","last_undegraded":"2024-08-18T13:39:40.431523+0000","last_fullsized":"2024-08-18T13:39:40.431523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457313+0000","last_change":"2024-08-18T13:39:39.456626+0000","last_active":"2024-08-18T13:39:39.457313+0000","last_peered":"2024-08-18T13:39:39.457313+0000","last_clean":"2024-08-18T13:39:39.457313+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:39:39.457313+0000","last_undegraded":"2024-08-18T13:39:39.457313+0000","last_fullsized":"2024-08-18T13:39:39.457313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450395+0000","last_change":"2024-08-18T13:39:35.417693+0000","last_active":"2024-08-18T13:39:39.450395+0000","last_peered":"2024-08-18T13:39:39.450395+0000","last_clean":"2024-08-18T13:39:39.450395+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:39:39.450395+0000","last_undegraded":"2024-08-18T13:39:39.450395+0000","last_fullsized":"2024-08-18T13:39:39.450395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:40:09.449886+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:40:09.449886+0000","last_peered":"2024-08-18T13:40:09.449886+0000","last_clean":"2024-08-18T13:40:09.449886+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:40:09.449886+0000","last_undegraded":"2024-08-18T13:40:09.449886+0000","last_fullsized":"2024-08-18T13:40:09.449886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4207,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.440639+0000","last_change":"2024-08-18T13:39:39.438064+0000","last_active":"2024-08-18T13:39:39.440639+0000","last_peered":"2024-08-18T13:39:39.440639+0000","last_clean":"2024-08-18T13:39:39.440639+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:39:39.440639+0000","last_undegraded":"2024-08-18T13:39:39.440639+0000","last_fullsized":"2024-08-18T13:39:39.440639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421839+0000","last_change":"2024-08-18T13:39:35.406398+0000","last_active":"2024-08-18T13:39:39.421839+0000","last_peered":"2024-08-18T13:39:39.421839+0000","last_clean":"2024-08-18T13:39:39.421839+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:39:39.421839+0000","last_undegraded":"2024-08-18T13:39:39.421839+0000","last_fullsized":"2024-08-18T13:39:39.421839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433257+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:39:40.433257+0000","last_peered":"2024-08-18T13:39:40.433257+0000","last_clean":"2024-08-18T13:39:40.433257+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:39:40.433257+0000","last_undegraded":"2024-08-18T13:39:40.433257+0000","last_fullsized":"2024-08-18T13:39:40.433257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444385+0000","last_change":"2024-08-18T13:39:39.439962+0000","last_active":"2024-08-18T13:39:39.444385+0000","last_peered":"2024-08-18T13:39:39.444385+0000","last_clean":"2024-08-18T13:39:39.444385+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:39:39.444385+0000","last_undegraded":"2024-08-18T13:39:39.444385+0000","last_fullsized":"2024-08-18T13:39:39.444385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432863+0000","last_change":"2024-08-18T13:39:35.411067+0000","last_active":"2024-08-18T13:39:39.432863+0000","last_peered":"2024-08-18T13:39:39.432863+0000","last_clean":"2024-08-18T13:39:39.432863+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:39:39.432863+0000","last_undegraded":"2024-08-18T13:39:39.432863+0000","last_fullsized":"2024-08-18T13:39:39.432863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422811+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:39:39.422811+0000","last_peered":"2024-08-18T13:39:39.422811+0000","last_clean":"2024-08-18T13:39:39.422811+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:39:39.422811+0000","last_undegraded":"2024-08-18T13:39:39.422811+0000","last_fullsized":"2024-08-18T13:39:39.422811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447069+0000","last_change":"2024-08-18T13:39:39.443938+0000","last_active":"2024-08-18T13:39:39.447069+0000","last_peered":"2024-08-18T13:39:39.447069+0000","last_clean":"2024-08-18T13:39:39.447069+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:39:39.447069+0000","last_undegraded":"2024-08-18T13:39:39.447069+0000","last_fullsized":"2024-08-18T13:39:39.447069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422769+0000","last_change":"2024-08-18T13:39:35.409517+0000","last_active":"2024-08-18T13:39:39.422769+0000","last_peered":"2024-08-18T13:39:39.422769+0000","last_clean":"2024-08-18T13:39:39.422769+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:39:39.422769+0000","last_undegraded":"2024-08-18T13:39:39.422769+0000","last_fullsized":"2024-08-18T13:39:39.422769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429841+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:39:39.429841+0000","last_peered":"2024-08-18T13:39:39.429841+0000","last_clean":"2024-08-18T13:39:39.429841+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:39:39.429841+0000","last_undegraded":"2024-08-18T13:39:39.429841+0000","last_fullsized":"2024-08-18T13:39:39.429841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450637+0000","last_change":"2024-08-18T13:39:39.449671+0000","last_active":"2024-08-18T13:39:39.450637+0000","last_peered":"2024-08-18T13:39:39.450637+0000","last_clean":"2024-08-18T13:39:39.450637+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:39:39.450637+0000","last_undegraded":"2024-08-18T13:39:39.450637+0000","last_fullsized":"2024-08-18T13:39:39.450637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454824+0000","last_change":"2024-08-18T13:39:39.453907+0000","last_active":"2024-08-18T13:39:39.454824+0000","last_peered":"2024-08-18T13:39:39.454824+0000","last_clean":"2024-08-18T13:39:39.454824+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:39:39.454824+0000","last_undegraded":"2024-08-18T13:39:39.454824+0000","last_fullsized":"2024-08-18T13:39:39.454824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422264+0000","last_change":"2024-08-18T13:39:35.401997+0000","last_active":"2024-08-18T13:39:39.422264+0000","last_peered":"2024-08-18T13:39:39.422264+0000","last_clean":"2024-08-18T13:39:39.422264+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:39:39.422264+0000","last_undegraded":"2024-08-18T13:39:39.422264+0000","last_fullsized":"2024-08-18T13:39:39.422264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418448+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:39:39.418448+0000","last_peered":"2024-08-18T13:39:39.418448+0000","last_clean":"2024-08-18T13:39:39.418448+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:39:39.418448+0000","last_undegraded":"2024-08-18T13:39:39.418448+0000","last_fullsized":"2024-08-18T13:39:39.418448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442977+0000","last_change":"2024-08-18T13:39:39.440811+0000","last_active":"2024-08-18T13:39:39.442977+0000","last_peered":"2024-08-18T13:39:39.442977+0000","last_clean":"2024-08-18T13:39:39.442977+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:39:39.442977+0000","last_undegraded":"2024-08-18T13:39:39.442977+0000","last_fullsized":"2024-08-18T13:39:39.442977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430210+0000","last_change":"2024-08-18T13:39:35.412127+0000","last_active":"2024-08-18T13:39:39.430210+0000","last_peered":"2024-08-18T13:39:39.430210+0000","last_clean":"2024-08-18T13:39:39.430210+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:39:39.430210+0000","last_undegraded":"2024-08-18T13:39:39.430210+0000","last_fullsized":"2024-08-18T13:39:39.430210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423847+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:39:39.423847+0000","last_peered":"2024-08-18T13:39:39.423847+0000","last_clean":"2024-08-18T13:39:39.423847+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:39:39.423847+0000","last_undegraded":"2024-08-18T13:39:39.423847+0000","last_fullsized":"2024-08-18T13:39:39.423847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429678+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:39:40.429678+0000","last_peered":"2024-08-18T13:39:40.429678+0000","last_clean":"2024-08-18T13:39:40.429678+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:39:40.429678+0000","last_undegraded":"2024-08-18T13:39:40.429678+0000","last_fullsized":"2024-08-18T13:39:40.429678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429574+0000","last_change":"2024-08-18T13:39:35.407995+0000","last_active":"2024-08-18T13:39:39.429574+0000","last_peered":"2024-08-18T13:39:39.429574+0000","last_clean":"2024-08-18T13:39:39.429574+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:39:39.429574+0000","last_undegraded":"2024-08-18T13:39:39.429574+0000","last_fullsized":"2024-08-18T13:39:39.429574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.438294+0000","last_change":"2024-08-18T13:39:39.437542+0000","last_active":"2024-08-18T13:39:39.438294+0000","last_peered":"2024-08-18T13:39:39.438294+0000","last_clean":"2024-08-18T13:39:39.438294+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:39:39.438294+0000","last_undegraded":"2024-08-18T13:39:39.438294+0000","last_fullsized":"2024-08-18T13:39:39.438294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434310+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:39:40.434310+0000","last_peered":"2024-08-18T13:39:40.434310+0000","last_clean":"2024-08-18T13:39:40.434310+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:39:40.434310+0000","last_undegraded":"2024-08-18T13:39:40.434310+0000","last_fullsized":"2024-08-18T13:39:40.434310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422760+0000","last_change":"2024-08-18T13:39:35.403759+0000","last_active":"2024-08-18T13:39:39.422760+0000","last_peered":"2024-08-18T13:39:39.422760+0000","last_clean":"2024-08-18T13:39:39.422760+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:39:39.422760+0000","last_undegraded":"2024-08-18T13:39:39.422760+0000","last_fullsized":"2024-08-18T13:39:39.422760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446432+0000","last_change":"2024-08-18T13:39:39.441201+0000","last_active":"2024-08-18T13:39:39.446432+0000","last_peered":"2024-08-18T13:39:39.446432+0000","last_clean":"2024-08-18T13:39:39.446432+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:39:39.446432+0000","last_undegraded":"2024-08-18T13:39:39.446432+0000","last_fullsized":"2024-08-18T13:39:39.446432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427616+0000","last_change":"2024-08-18T13:39:35.413292+0000","last_active":"2024-08-18T13:39:39.427616+0000","last_peered":"2024-08-18T13:39:39.427616+0000","last_clean":"2024-08-18T13:39:39.427616+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:39:39.427616+0000","last_undegraded":"2024-08-18T13:39:39.427616+0000","last_fullsized":"2024-08-18T13:39:39.427616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415812+0000","last_change":"2024-08-18T13:39:35.401196+0000","last_active":"2024-08-18T13:39:39.415812+0000","last_peered":"2024-08-18T13:39:39.415812+0000","last_clean":"2024-08-18T13:39:39.415812+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:39:39.415812+0000","last_undegraded":"2024-08-18T13:39:39.415812+0000","last_fullsized":"2024-08-18T13:39:39.415812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417577+0000","last_change":"2024-08-18T13:39:35.406603+0000","last_active":"2024-08-18T13:39:39.417577+0000","last_peered":"2024-08-18T13:39:39.417577+0000","last_clean":"2024-08-18T13:39:39.417577+0000","last_became_active":"2024-08-18T13:39:35.406331+0000","last_became_peered":"2024-08-18T13:39:35.406331+0000","last_unstale":"2024-08-18T13:39:39.417577+0000","last_undegraded":"2024-08-18T13:39:39.417577+0000","last_fullsized":"2024-08-18T13:39:39.417577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:21:12.232172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429494+0000","last_change":"2024-08-18T13:39:35.404051+0000","last_active":"2024-08-18T13:39:39.429494+0000","last_peered":"2024-08-18T13:39:39.429494+0000","last_clean":"2024-08-18T13:39:39.429494+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:39:39.429494+0000","last_undegraded":"2024-08-18T13:39:39.429494+0000","last_fullsized":"2024-08-18T13:39:39.429494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422959+0000","last_change":"2024-08-18T13:39:35.409288+0000","last_active":"2024-08-18T13:39:39.422959+0000","last_peered":"2024-08-18T13:39:39.422959+0000","last_clean":"2024-08-18T13:39:39.422959+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:39:39.422959+0000","last_undegraded":"2024-08-18T13:39:39.422959+0000","last_fullsized":"2024-08-18T13:39:39.422959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424121+0000","last_change":"2024-08-18T13:39:35.403051+0000","last_active":"2024-08-18T13:39:39.424121+0000","last_peered":"2024-08-18T13:39:39.424121+0000","last_clean":"2024-08-18T13:39:39.424121+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:39:39.424121+0000","last_undegraded":"2024-08-18T13:39:39.424121+0000","last_fullsized":"2024-08-18T13:39:39.424121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430049+0000","last_change":"2024-08-18T13:39:35.412281+0000","last_active":"2024-08-18T13:39:39.430049+0000","last_peered":"2024-08-18T13:39:39.430049+0000","last_clean":"2024-08-18T13:39:39.430049+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:39:39.430049+0000","last_undegraded":"2024-08-18T13:39:39.430049+0000","last_fullsized":"2024-08-18T13:39:39.430049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424466+0000","last_change":"2024-08-18T13:39:35.409220+0000","last_active":"2024-08-18T13:39:39.424466+0000","last_peered":"2024-08-18T13:39:39.424466+0000","last_clean":"2024-08-18T13:39:39.424466+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:39:39.424466+0000","last_undegraded":"2024-08-18T13:39:39.424466+0000","last_fullsized":"2024-08-18T13:39:39.424466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418253+0000","last_change":"2024-08-18T13:39:35.401759+0000","last_active":"2024-08-18T13:39:39.418253+0000","last_peered":"2024-08-18T13:39:39.418253+0000","last_clean":"2024-08-18T13:39:39.418253+0000","last_became_active":"2024-08-18T13:39:35.401476+0000","last_became_peered":"2024-08-18T13:39:35.401476+0000","last_unstale":"2024-08-18T13:39:39.418253+0000","last_undegraded":"2024-08-18T13:39:39.418253+0000","last_fullsized":"2024-08-18T13:39:39.418253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:04:43.997597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423669+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.423669+0000","last_peered":"2024-08-18T13:39:39.423669+0000","last_clean":"2024-08-18T13:39:39.423669+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:39:39.423669+0000","last_undegraded":"2024-08-18T13:39:39.423669+0000","last_fullsized":"2024-08-18T13:39:39.423669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417410+0000","last_change":"2024-08-18T13:39:35.407218+0000","last_active":"2024-08-18T13:39:39.417410+0000","last_peered":"2024-08-18T13:39:39.417410+0000","last_clean":"2024-08-18T13:39:39.417410+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:39:39.417410+0000","last_undegraded":"2024-08-18T13:39:39.417410+0000","last_fullsized":"2024-08-18T13:39:39.417410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429387+0000","last_change":"2024-08-18T13:39:35.407339+0000","last_active":"2024-08-18T13:39:39.429387+0000","last_peered":"2024-08-18T13:39:39.429387+0000","last_clean":"2024-08-18T13:39:39.429387+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:39:39.429387+0000","last_undegraded":"2024-08-18T13:39:39.429387+0000","last_fullsized":"2024-08-18T13:39:39.429387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429062+0000","last_change":"2024-08-18T13:39:35.408647+0000","last_active":"2024-08-18T13:39:39.429062+0000","last_peered":"2024-08-18T13:39:39.429062+0000","last_clean":"2024-08-18T13:39:39.429062+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:39:39.429062+0000","last_undegraded":"2024-08-18T13:39:39.429062+0000","last_fullsized":"2024-08-18T13:39:39.429062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450837+0000","last_change":"2024-08-18T13:39:35.418000+0000","last_active":"2024-08-18T13:39:39.450837+0000","last_peered":"2024-08-18T13:39:39.450837+0000","last_clean":"2024-08-18T13:39:39.450837+0000","last_became_active":"2024-08-18T13:39:35.417661+0000","last_became_peered":"2024-08-18T13:39:35.417661+0000","last_unstale":"2024-08-18T13:39:39.450837+0000","last_undegraded":"2024-08-18T13:39:39.450837+0000","last_fullsized":"2024-08-18T13:39:39.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:53:44.739585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416124+0000","last_change":"2024-08-18T13:39:35.405425+0000","last_active":"2024-08-18T13:39:39.416124+0000","last_peered":"2024-08-18T13:39:39.416124+0000","last_clean":"2024-08-18T13:39:39.416124+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:39:39.416124+0000","last_undegraded":"2024-08-18T13:39:39.416124+0000","last_fullsized":"2024-08-18T13:39:39.416124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423886+0000","last_change":"2024-08-18T13:39:35.400889+0000","last_active":"2024-08-18T13:39:39.423886+0000","last_peered":"2024-08-18T13:39:39.423886+0000","last_clean":"2024-08-18T13:39:39.423886+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:39:39.423886+0000","last_undegraded":"2024-08-18T13:39:39.423886+0000","last_fullsized":"2024-08-18T13:39:39.423886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418057+0000","last_change":"2024-08-18T13:39:35.407634+0000","last_active":"2024-08-18T13:39:39.418057+0000","last_peered":"2024-08-18T13:39:39.418057+0000","last_clean":"2024-08-18T13:39:39.418057+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:39:39.418057+0000","last_undegraded":"2024-08-18T13:39:39.418057+0000","last_fullsized":"2024-08-18T13:39:39.418057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422473+0000","last_change":"2024-08-18T13:39:35.397509+0000","last_active":"2024-08-18T13:39:39.422473+0000","last_peered":"2024-08-18T13:39:39.422473+0000","last_clean":"2024-08-18T13:39:39.422473+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:39:39.422473+0000","last_undegraded":"2024-08-18T13:39:39.422473+0000","last_fullsized":"2024-08-18T13:39:39.422473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425330+0000","last_change":"2024-08-18T13:39:35.404533+0000","last_active":"2024-08-18T13:39:39.425330+0000","last_peered":"2024-08-18T13:39:39.425330+0000","last_clean":"2024-08-18T13:39:39.425330+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:39:39.425330+0000","last_undegraded":"2024-08-18T13:39:39.425330+0000","last_fullsized":"2024-08-18T13:39:39.425330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425567+0000","last_change":"2024-08-18T13:39:35.404839+0000","last_active":"2024-08-18T13:39:39.425567+0000","last_peered":"2024-08-18T13:39:39.425567+0000","last_clean":"2024-08-18T13:39:39.425567+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:39:39.425567+0000","last_undegraded":"2024-08-18T13:39:39.425567+0000","last_fullsized":"2024-08-18T13:39:39.425567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421633+0000","last_change":"2024-08-18T13:39:35.405867+0000","last_active":"2024-08-18T13:39:39.421633+0000","last_peered":"2024-08-18T13:39:39.421633+0000","last_clean":"2024-08-18T13:39:39.421633+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:39:39.421633+0000","last_undegraded":"2024-08-18T13:39:39.421633+0000","last_fullsized":"2024-08-18T13:39:39.421633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429697+0000","last_change":"2024-08-18T13:39:35.411173+0000","last_active":"2024-08-18T13:39:39.429697+0000","last_peered":"2024-08-18T13:39:39.429697+0000","last_clean":"2024-08-18T13:39:39.429697+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:39:39.429697+0000","last_undegraded":"2024-08-18T13:39:39.429697+0000","last_fullsized":"2024-08-18T13:39:39.429697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429962+0000","last_change":"2024-08-18T13:39:35.418013+0000","last_active":"2024-08-18T13:39:39.429962+0000","last_peered":"2024-08-18T13:39:39.429962+0000","last_clean":"2024-08-18T13:39:39.429962+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:39:39.429962+0000","last_undegraded":"2024-08-18T13:39:39.429962+0000","last_fullsized":"2024-08-18T13:39:39.429962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450803+0000","last_change":"2024-08-18T13:39:35.421633+0000","last_active":"2024-08-18T13:39:39.450803+0000","last_peered":"2024-08-18T13:39:39.450803+0000","last_clean":"2024-08-18T13:39:39.450803+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:39:39.450803+0000","last_undegraded":"2024-08-18T13:39:39.450803+0000","last_fullsized":"2024-08-18T13:39:39.450803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425234+0000","last_change":"2024-08-18T13:39:35.411443+0000","last_active":"2024-08-18T13:39:39.425234+0000","last_peered":"2024-08-18T13:39:39.425234+0000","last_clean":"2024-08-18T13:39:39.425234+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:39:39.425234+0000","last_undegraded":"2024-08-18T13:39:39.425234+0000","last_fullsized":"2024-08-18T13:39:39.425234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450366+0000","last_change":"2024-08-18T13:39:35.415548+0000","last_active":"2024-08-18T13:39:39.450366+0000","last_peered":"2024-08-18T13:39:39.450366+0000","last_clean":"2024-08-18T13:39:39.450366+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:39:39.450366+0000","last_undegraded":"2024-08-18T13:39:39.450366+0000","last_fullsized":"2024-08-18T13:39:39.450366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417246+0000","last_change":"2024-08-18T13:39:35.410795+0000","last_active":"2024-08-18T13:39:39.417246+0000","last_peered":"2024-08-18T13:39:39.417246+0000","last_clean":"2024-08-18T13:39:39.417246+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:39:39.417246+0000","last_undegraded":"2024-08-18T13:39:39.417246+0000","last_fullsized":"2024-08-18T13:39:39.417246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450710+0000","last_change":"2024-08-18T13:39:35.417550+0000","last_active":"2024-08-18T13:39:39.450710+0000","last_peered":"2024-08-18T13:39:39.450710+0000","last_clean":"2024-08-18T13:39:39.450710+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:39:39.450710+0000","last_undegraded":"2024-08-18T13:39:39.450710+0000","last_fullsized":"2024-08-18T13:39:39.450710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422555+0000","last_change":"2024-08-18T13:39:35.410002+0000","last_active":"2024-08-18T13:39:39.422555+0000","last_peered":"2024-08-18T13:39:39.422555+0000","last_clean":"2024-08-18T13:39:39.422555+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:39:39.422555+0000","last_undegraded":"2024-08-18T13:39:39.422555+0000","last_fullsized":"2024-08-18T13:39:39.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427751+0000","last_change":"2024-08-18T13:39:35.406627+0000","last_active":"2024-08-18T13:39:39.427751+0000","last_peered":"2024-08-18T13:39:39.427751+0000","last_clean":"2024-08-18T13:39:39.427751+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:39:39.427751+0000","last_undegraded":"2024-08-18T13:39:39.427751+0000","last_fullsized":"2024-08-18T13:39:39.427751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423078+0000","last_change":"2024-08-18T13:39:35.405750+0000","last_active":"2024-08-18T13:39:39.423078+0000","last_peered":"2024-08-18T13:39:39.423078+0000","last_clean":"2024-08-18T13:39:39.423078+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:39:39.423078+0000","last_undegraded":"2024-08-18T13:39:39.423078+0000","last_fullsized":"2024-08-18T13:39:39.423078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450591+0000","last_change":"2024-08-18T13:39:35.417195+0000","last_active":"2024-08-18T13:39:39.450591+0000","last_peered":"2024-08-18T13:39:39.450591+0000","last_clean":"2024-08-18T13:39:39.450591+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:39:39.450591+0000","last_undegraded":"2024-08-18T13:39:39.450591+0000","last_fullsized":"2024-08-18T13:39:39.450591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4993,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":856064,"data_stored":693001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":270336,"data_stored":102189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":258048,"data_stored":102063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]}]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":102223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":864256,"data_stored":693057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":262144,"data_stored":102153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:40:14.802 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:40:14.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d4ce23700 1 Processor -- start 2024-08-18T13:40:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d4ce23700 1 -- start start 2024-08-18T13:40:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d4ce23700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d480c22a0 0x7f2d4814b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d4ce23700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4806de00 con 0x7f2d480be5a0 2024-08-18T13:40:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d4ce23700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d480c2da0 con 0x7f2d4806e4c0 2024-08-18T13:40:14.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d4ce23700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4806fac0 con 0x7f2d480c22a0 2024-08-18T13:40:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d477fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2d480be5a0 0x7f2d480c7fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53924/0 (socket says 172.21.15.118:53924) 2024-08-18T13:40:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d477fe700 1 -- 172.21.15.118:0/3627830049 learned_addr learned my addr 172.21.15.118:0/3627830049 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d46ffd700 1 --2- 172.21.15.118:0/3627830049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d480c22a0 0x7f2d4814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d467fc700 1 -- 172.21.15.118:0/3627830049 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638484659 0 0) 0x7f2d4806de00 con 0x7f2d480be5a0 2024-08-18T13:40:14.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d467fc700 1 -- 172.21.15.118:0/3627830049 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d34003570 con 0x7f2d480be5a0 2024-08-18T13:40:14.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d467fc700 1 -- 172.21.15.118:0/3627830049 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725070557 0 0) 0x7f2d480c2da0 con 0x7f2d4806e4c0 2024-08-18T13:40:14.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d46ffd700 1 -- 172.21.15.118:0/3627830049 >> v1:172.21.15.178:6789/0 conn(0x7f2d4806e4c0 legacy=0x7f2d48151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d46ffd700 1 -- 172.21.15.118:0/3627830049 >> v1:172.21.15.118:6789/0 conn(0x7f2d480be5a0 legacy=0x7f2d480c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.902+0000 7f2d46ffd700 1 -- 172.21.15.118:0/3627830049 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480c3080 con 0x7f2d480c22a0 2024-08-18T13:40:14.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d46ffd700 1 --2- 172.21.15.118:0/3627830049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d480c22a0 0x7f2d4814b940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009e90 tx=0x7f2d30009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b4d33b985cdaa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d467fc700 1 -- 172.21.15.118:0/3627830049 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000e070 con 0x7f2d480c22a0 2024-08-18T13:40:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d467fc700 1 -- 172.21.15.118:0/3627830049 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3000bc30 con 0x7f2d480c22a0 2024-08-18T13:40:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d467fc700 1 -- 172.21.15.118:0/3627830049 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30012430 con 0x7f2d480c22a0 2024-08-18T13:40:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- 172.21.15.118:0/3627830049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d480c22a0 msgr2=0x7f2d4814b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/3627830049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d480c22a0 0x7f2d4814b940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009e90 tx=0x7f2d30009780 comp rx=0 tx=0).stop 2024-08-18T13:40:14.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- 172.21.15.118:0/3627830049 shutdown_connections 2024-08-18T13:40:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/3627830049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d480c22a0 0x7f2d4814b940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- 172.21.15.118:0/3627830049 >> 172.21.15.118:0/3627830049 conn(0x7f2d4806c020 msgr2=0x7f2d480c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- 172.21.15.118:0/3627830049 shutdown_connections 2024-08-18T13:40:14.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- 172.21.15.118:0/3627830049 wait complete. 2024-08-18T13:40:14.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 Processor -- start 2024-08-18T13:40:14.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- start start 2024-08-18T13:40:14.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d4806e4c0 0x7f2d480c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d480be5a0 0x7f2d4814cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d480c22a0 0x7f2d4814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d48146a60 con 0x7f2d480be5a0 2024-08-18T13:40:14.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d481475a0 con 0x7f2d480c22a0 2024-08-18T13:40:14.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d4ce23700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4814ef40 con 0x7f2d4806e4c0 2024-08-18T13:40:14.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d480be5a0 0x7f2d4814cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d477fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d4806e4c0 0x7f2d480c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d480be5a0 0x7f2d4814cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51956/0 (socket says 172.21.15.118:51956) 2024-08-18T13:40:14.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d46ffd700 1 -- 172.21.15.118:0/1619698587 learned_addr learned my addr 172.21.15.118:0/1619698587 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:14.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.906+0000 7f2d47fff700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d480c22a0 0x7f2d4814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d46ffd700 1 -- 172.21.15.118:0/1619698587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d4806e4c0 msgr2=0x7f2d480c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d46ffd700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d4806e4c0 0x7f2d480c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.924 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d46ffd700 1 -- 172.21.15.118:0/1619698587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d480c22a0 msgr2=0x7f2d4814d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d46ffd700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d480c22a0 0x7f2d4814d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d46ffd700 1 -- 172.21.15.118:0/1619698587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30009a20 con 0x7f2d480be5a0 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d46ffd700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d480be5a0 0x7f2d4814cc70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d300097c0 tx=0x7f2d30019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d3f7fe700 1 -- 172.21.15.118:0/1619698587 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30012e60 con 0x7f2d480be5a0 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d3f7fe700 1 -- 172.21.15.118:0/1619698587 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3001b420 con 0x7f2d480be5a0 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d48146c70 con 0x7f2d480be5a0 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d3f7fe700 1 -- 172.21.15.118:0/1619698587 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3001b7e0 con 0x7f2d480be5a0 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d48147230 con 0x7f2d480be5a0 2024-08-18T13:40:14.925 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d480713a0 con 0x7f2d480be5a0 2024-08-18T13:40:14.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d3f7fe700 1 -- 172.21.15.118:0/1619698587 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2d3001bb90 con 0x7f2d480be5a0 2024-08-18T13:40:14.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d3f7fe700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d28060860 0x7f2d28062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:14.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d3f7fe700 1 -- 172.21.15.118:0/1619698587 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f2d3007eb00 con 0x7f2d480be5a0 2024-08-18T13:40:14.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.910+0000 7f2d477fe700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d28060860 0x7f2d28062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:14.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.914+0000 7f2d477fe700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d28060860 0x7f2d28062d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d38007fd0 tx=0x7f2d38009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:14.926 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:14.918+0000 7f2d3f7fe700 1 -- 172.21.15.118:0/1619698587 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2d3004ec60 con 0x7f2d480be5a0 2024-08-18T13:40:15.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.078+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d4806cf00 con 0x7f2d28060860 2024-08-18T13:40:15.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.094+0000 7f2d3f7fe700 1 -- 172.21.15.118:0/1619698587 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345233 (secure 0 0 0) 0x7f2d4806cf00 con 0x7f2d28060860 2024-08-18T13:40:15.106 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d28060860 msgr2=0x7f2d28062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d28060860 0x7f2d28062d10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d38007fd0 tx=0x7f2d38009040 comp rx=0 tx=0).stop 2024-08-18T13:40:15.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d480be5a0 msgr2=0x7f2d4814cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d480be5a0 0x7f2d4814cc70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d300097c0 tx=0x7f2d30019000 comp rx=0 tx=0).stop 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 shutdown_connections 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d28060860 0x7f2d28062d10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d480c22a0 0x7f2d4814d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d480be5a0 0x7f2d4814cc70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 --2- 172.21.15.118:0/1619698587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d4806e4c0 0x7f2d480c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 >> 172.21.15.118:0/1619698587 conn(0x7f2d4806c020 msgr2=0x7f2d480c4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 shutdown_connections 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.102+0000 7f2d4ce23700 1 -- 172.21.15.118:0/1619698587 wait complete. 2024-08-18T13:40:15.112 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:40:15.119 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-08-18T13:40:13.210835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595361,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":209,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415368,"kb_used_data":3240,"kb_used_omap":13,"kb_used_meta":2412018,"kb_avail":752559352,"statfs":{"total":773094113280,"available":770620776448,"internally_reserved":0,"allocated":3317760,"data_stored":2007226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2469907136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":445,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005690"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433629+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:39:40.433629+0000","last_peered":"2024-08-18T13:39:40.433629+0000","last_clean":"2024-08-18T13:39:40.433629+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:39:40.433629+0000","last_undegraded":"2024-08-18T13:39:40.433629+0000","last_fullsized":"2024-08-18T13:39:40.433629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422872+0000","last_change":"2024-08-18T13:39:35.405966+0000","last_active":"2024-08-18T13:39:39.422872+0000","last_peered":"2024-08-18T13:39:39.422872+0000","last_clean":"2024-08-18T13:39:39.422872+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:39:39.422872+0000","last_undegraded":"2024-08-18T13:39:39.422872+0000","last_fullsized":"2024-08-18T13:39:39.422872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454715+0000","last_change":"2024-08-18T13:39:39.453674+0000","last_active":"2024-08-18T13:39:39.454715+0000","last_peered":"2024-08-18T13:39:39.454715+0000","last_clean":"2024-08-18T13:39:39.454715+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:39:39.454715+0000","last_undegraded":"2024-08-18T13:39:39.454715+0000","last_fullsized":"2024-08-18T13:39:39.454715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434410+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:39:40.434410+0000","last_peered":"2024-08-18T13:39:40.434410+0000","last_clean":"2024-08-18T13:39:40.434410+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:39:40.434410+0000","last_undegraded":"2024-08-18T13:39:40.434410+0000","last_fullsized":"2024-08-18T13:39:40.434410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432862+0000","last_change":"2024-08-18T13:39:35.408917+0000","last_active":"2024-08-18T13:39:39.432862+0000","last_peered":"2024-08-18T13:39:39.432862+0000","last_clean":"2024-08-18T13:39:39.432862+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:39:39.432862+0000","last_undegraded":"2024-08-18T13:39:39.432862+0000","last_fullsized":"2024-08-18T13:39:39.432862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447766+0000","last_change":"2024-08-18T13:39:39.445388+0000","last_active":"2024-08-18T13:39:39.447766+0000","last_peered":"2024-08-18T13:39:39.447766+0000","last_clean":"2024-08-18T13:39:39.447766+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:39:39.447766+0000","last_undegraded":"2024-08-18T13:39:39.447766+0000","last_fullsized":"2024-08-18T13:39:39.447766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430049+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:39:40.430049+0000","last_peered":"2024-08-18T13:39:40.430049+0000","last_clean":"2024-08-18T13:39:40.430049+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:39:40.430049+0000","last_undegraded":"2024-08-18T13:39:40.430049+0000","last_fullsized":"2024-08-18T13:39:40.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422649+0000","last_change":"2024-08-18T13:39:35.399188+0000","last_active":"2024-08-18T13:39:39.422649+0000","last_peered":"2024-08-18T13:39:39.422649+0000","last_clean":"2024-08-18T13:39:39.422649+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:39:39.422649+0000","last_undegraded":"2024-08-18T13:39:39.422649+0000","last_fullsized":"2024-08-18T13:39:39.422649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447285+0000","last_change":"2024-08-18T13:39:39.440922+0000","last_active":"2024-08-18T13:39:39.447285+0000","last_peered":"2024-08-18T13:39:39.447285+0000","last_clean":"2024-08-18T13:39:39.447285+0000","last_became_active":"2024-08-18T13:39:39.440426+0000","last_became_peered":"2024-08-18T13:39:39.440426+0000","last_unstale":"2024-08-18T13:39:39.447285+0000","last_undegraded":"2024-08-18T13:39:39.447285+0000","last_fullsized":"2024-08-18T13:39:39.447285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:23:00.376805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.428764+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:39:40.428764+0000","last_peered":"2024-08-18T13:39:40.428764+0000","last_clean":"2024-08-18T13:39:40.428764+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:39:40.428764+0000","last_undegraded":"2024-08-18T13:39:40.428764+0000","last_fullsized":"2024-08-18T13:39:40.428764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423184+0000","last_change":"2024-08-18T13:39:35.400038+0000","last_active":"2024-08-18T13:39:39.423184+0000","last_peered":"2024-08-18T13:39:39.423184+0000","last_clean":"2024-08-18T13:39:39.423184+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:39:39.423184+0000","last_undegraded":"2024-08-18T13:39:39.423184+0000","last_fullsized":"2024-08-18T13:39:39.423184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437377+0000","last_change":"2024-08-18T13:39:39.436221+0000","last_active":"2024-08-18T13:39:39.437377+0000","last_peered":"2024-08-18T13:39:39.437377+0000","last_clean":"2024-08-18T13:39:39.437377+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:39:39.437377+0000","last_undegraded":"2024-08-18T13:39:39.437377+0000","last_fullsized":"2024-08-18T13:39:39.437377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428676+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.428676+0000","last_peered":"2024-08-18T13:39:39.428676+0000","last_clean":"2024-08-18T13:39:39.428676+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:39:39.428676+0000","last_undegraded":"2024-08-18T13:39:39.428676+0000","last_fullsized":"2024-08-18T13:39:39.428676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430090+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:39:40.430090+0000","last_peered":"2024-08-18T13:39:40.430090+0000","last_clean":"2024-08-18T13:39:40.430090+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:39:40.430090+0000","last_undegraded":"2024-08-18T13:39:40.430090+0000","last_fullsized":"2024-08-18T13:39:40.430090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.443080+0000","last_change":"2024-08-18T13:39:39.439633+0000","last_active":"2024-08-18T13:39:39.443080+0000","last_peered":"2024-08-18T13:39:39.443080+0000","last_clean":"2024-08-18T13:39:39.443080+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:39:39.443080+0000","last_undegraded":"2024-08-18T13:39:39.443080+0000","last_fullsized":"2024-08-18T13:39:39.443080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423607+0000","last_change":"2024-08-18T13:39:35.407947+0000","last_active":"2024-08-18T13:39:39.423607+0000","last_peered":"2024-08-18T13:39:39.423607+0000","last_clean":"2024-08-18T13:39:39.423607+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:39:39.423607+0000","last_undegraded":"2024-08-18T13:39:39.423607+0000","last_fullsized":"2024-08-18T13:39:39.423607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429156+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:39:39.429156+0000","last_peered":"2024-08-18T13:39:39.429156+0000","last_clean":"2024-08-18T13:39:39.429156+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:39:39.429156+0000","last_undegraded":"2024-08-18T13:39:39.429156+0000","last_fullsized":"2024-08-18T13:39:39.429156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442892+0000","last_change":"2024-08-18T13:39:39.439913+0000","last_active":"2024-08-18T13:39:39.442892+0000","last_peered":"2024-08-18T13:39:39.442892+0000","last_clean":"2024-08-18T13:39:39.442892+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:39:39.442892+0000","last_undegraded":"2024-08-18T13:39:39.442892+0000","last_fullsized":"2024-08-18T13:39:39.442892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430332+0000","last_change":"2024-08-18T13:39:35.403254+0000","last_active":"2024-08-18T13:39:39.430332+0000","last_peered":"2024-08-18T13:39:39.430332+0000","last_clean":"2024-08-18T13:39:39.430332+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:39:39.430332+0000","last_undegraded":"2024-08-18T13:39:39.430332+0000","last_fullsized":"2024-08-18T13:39:39.430332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422689+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:39:39.422689+0000","last_peered":"2024-08-18T13:39:39.422689+0000","last_clean":"2024-08-18T13:39:39.422689+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:39:39.422689+0000","last_undegraded":"2024-08-18T13:39:39.422689+0000","last_fullsized":"2024-08-18T13:39:39.422689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457822+0000","last_change":"2024-08-18T13:39:39.457250+0000","last_active":"2024-08-18T13:39:39.457822+0000","last_peered":"2024-08-18T13:39:39.457822+0000","last_clean":"2024-08-18T13:39:39.457822+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:39:39.457822+0000","last_undegraded":"2024-08-18T13:39:39.457822+0000","last_fullsized":"2024-08-18T13:39:39.457822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425047+0000","last_change":"2024-08-18T13:39:35.401612+0000","last_active":"2024-08-18T13:39:39.425047+0000","last_peered":"2024-08-18T13:39:39.425047+0000","last_clean":"2024-08-18T13:39:39.425047+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:39:39.425047+0000","last_undegraded":"2024-08-18T13:39:39.425047+0000","last_fullsized":"2024-08-18T13:39:39.425047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450234+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:39:39.450234+0000","last_peered":"2024-08-18T13:39:39.450234+0000","last_clean":"2024-08-18T13:39:39.450234+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:39:39.450234+0000","last_undegraded":"2024-08-18T13:39:39.450234+0000","last_fullsized":"2024-08-18T13:39:39.450234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447688+0000","last_change":"2024-08-18T13:39:39.443503+0000","last_active":"2024-08-18T13:39:39.447688+0000","last_peered":"2024-08-18T13:39:39.447688+0000","last_clean":"2024-08-18T13:39:39.447688+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:39:39.447688+0000","last_undegraded":"2024-08-18T13:39:39.447688+0000","last_fullsized":"2024-08-18T13:39:39.447688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432976+0000","last_change":"2024-08-18T13:39:35.407907+0000","last_active":"2024-08-18T13:39:39.432976+0000","last_peered":"2024-08-18T13:39:39.432976+0000","last_clean":"2024-08-18T13:39:39.432976+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:39:39.432976+0000","last_undegraded":"2024-08-18T13:39:39.432976+0000","last_fullsized":"2024-08-18T13:39:39.432976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429181+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:39:40.429181+0000","last_peered":"2024-08-18T13:39:40.429181+0000","last_clean":"2024-08-18T13:39:40.429181+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:39:40.429181+0000","last_undegraded":"2024-08-18T13:39:40.429181+0000","last_fullsized":"2024-08-18T13:39:40.429181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454767+0000","last_change":"2024-08-18T13:39:39.453138+0000","last_active":"2024-08-18T13:39:39.454767+0000","last_peered":"2024-08-18T13:39:39.454767+0000","last_clean":"2024-08-18T13:39:39.454767+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:39:39.454767+0000","last_undegraded":"2024-08-18T13:39:39.454767+0000","last_fullsized":"2024-08-18T13:39:39.454767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424552+0000","last_change":"2024-08-18T13:39:35.404453+0000","last_active":"2024-08-18T13:39:39.424552+0000","last_peered":"2024-08-18T13:39:39.424552+0000","last_clean":"2024-08-18T13:39:39.424552+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:39:39.424552+0000","last_undegraded":"2024-08-18T13:39:39.424552+0000","last_fullsized":"2024-08-18T13:39:39.424552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:41.051281+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:39:41.051281+0000","last_peered":"2024-08-18T13:39:41.051281+0000","last_clean":"2024-08-18T13:39:41.051281+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:39:41.051281+0000","last_undegraded":"2024-08-18T13:39:41.051281+0000","last_fullsized":"2024-08-18T13:39:41.051281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450645+0000","last_change":"2024-08-18T13:39:39.449851+0000","last_active":"2024-08-18T13:39:39.450645+0000","last_peered":"2024-08-18T13:39:39.450645+0000","last_clean":"2024-08-18T13:39:39.450645+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:39:39.450645+0000","last_undegraded":"2024-08-18T13:39:39.450645+0000","last_fullsized":"2024-08-18T13:39:39.450645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424459+0000","last_change":"2024-08-18T13:39:35.400604+0000","last_active":"2024-08-18T13:39:39.424459+0000","last_peered":"2024-08-18T13:39:39.424459+0000","last_clean":"2024-08-18T13:39:39.424459+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:39:39.424459+0000","last_undegraded":"2024-08-18T13:39:39.424459+0000","last_fullsized":"2024-08-18T13:39:39.424459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450871+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:39:39.450871+0000","last_peered":"2024-08-18T13:39:39.450871+0000","last_clean":"2024-08-18T13:39:39.450871+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:39:39.450871+0000","last_undegraded":"2024-08-18T13:39:39.450871+0000","last_fullsized":"2024-08-18T13:39:39.450871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454096+0000","last_change":"2024-08-18T13:39:39.453599+0000","last_active":"2024-08-18T13:39:39.454096+0000","last_peered":"2024-08-18T13:39:39.454096+0000","last_clean":"2024-08-18T13:39:39.454096+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:39:39.454096+0000","last_undegraded":"2024-08-18T13:39:39.454096+0000","last_fullsized":"2024-08-18T13:39:39.454096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423160+0000","last_change":"2024-08-18T13:39:35.410167+0000","last_active":"2024-08-18T13:39:39.423160+0000","last_peered":"2024-08-18T13:39:39.423160+0000","last_clean":"2024-08-18T13:39:39.423160+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:39:39.423160+0000","last_undegraded":"2024-08-18T13:39:39.423160+0000","last_fullsized":"2024-08-18T13:39:39.423160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424551+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:39:39.424551+0000","last_peered":"2024-08-18T13:39:39.424551+0000","last_clean":"2024-08-18T13:39:39.424551+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:39:39.424551+0000","last_undegraded":"2024-08-18T13:39:39.424551+0000","last_fullsized":"2024-08-18T13:39:39.424551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444517+0000","last_change":"2024-08-18T13:39:39.439948+0000","last_active":"2024-08-18T13:39:39.444517+0000","last_peered":"2024-08-18T13:39:39.444517+0000","last_clean":"2024-08-18T13:39:39.444517+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:39:39.444517+0000","last_undegraded":"2024-08-18T13:39:39.444517+0000","last_fullsized":"2024-08-18T13:39:39.444517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422894+0000","last_change":"2024-08-18T13:39:35.407829+0000","last_active":"2024-08-18T13:39:39.422894+0000","last_peered":"2024-08-18T13:39:39.422894+0000","last_clean":"2024-08-18T13:39:39.422894+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:39:39.422894+0000","last_undegraded":"2024-08-18T13:39:39.422894+0000","last_fullsized":"2024-08-18T13:39:39.422894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417877+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:39:39.417877+0000","last_peered":"2024-08-18T13:39:39.417877+0000","last_clean":"2024-08-18T13:39:39.417877+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:39:39.417877+0000","last_undegraded":"2024-08-18T13:39:39.417877+0000","last_fullsized":"2024-08-18T13:39:39.417877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445479+0000","last_change":"2024-08-18T13:39:39.442372+0000","last_active":"2024-08-18T13:39:39.445479+0000","last_peered":"2024-08-18T13:39:39.445479+0000","last_clean":"2024-08-18T13:39:39.445479+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:39:39.445479+0000","last_undegraded":"2024-08-18T13:39:39.445479+0000","last_fullsized":"2024-08-18T13:39:39.445479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421862+0000","last_change":"2024-08-18T13:39:35.400384+0000","last_active":"2024-08-18T13:39:39.421862+0000","last_peered":"2024-08-18T13:39:39.421862+0000","last_clean":"2024-08-18T13:39:39.421862+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:39:39.421862+0000","last_undegraded":"2024-08-18T13:39:39.421862+0000","last_fullsized":"2024-08-18T13:39:39.421862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415832+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:39:39.415832+0000","last_peered":"2024-08-18T13:39:39.415832+0000","last_clean":"2024-08-18T13:39:39.415832+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:39:39.415832+0000","last_undegraded":"2024-08-18T13:39:39.415832+0000","last_fullsized":"2024-08-18T13:39:39.415832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446859+0000","last_change":"2024-08-18T13:39:39.446401+0000","last_active":"2024-08-18T13:39:39.446859+0000","last_peered":"2024-08-18T13:39:39.446859+0000","last_clean":"2024-08-18T13:39:39.446859+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:39:39.446859+0000","last_undegraded":"2024-08-18T13:39:39.446859+0000","last_fullsized":"2024-08-18T13:39:39.446859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424307+0000","last_change":"2024-08-18T13:39:35.410127+0000","last_active":"2024-08-18T13:39:39.424307+0000","last_peered":"2024-08-18T13:39:39.424307+0000","last_clean":"2024-08-18T13:39:39.424307+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:39:39.424307+0000","last_undegraded":"2024-08-18T13:39:39.424307+0000","last_fullsized":"2024-08-18T13:39:39.424307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450338+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:39:39.450338+0000","last_peered":"2024-08-18T13:39:39.450338+0000","last_clean":"2024-08-18T13:39:39.450338+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:39:39.450338+0000","last_undegraded":"2024-08-18T13:39:39.450338+0000","last_fullsized":"2024-08-18T13:39:39.450338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454861+0000","last_change":"2024-08-18T13:39:39.453901+0000","last_active":"2024-08-18T13:39:39.454861+0000","last_peered":"2024-08-18T13:39:39.454861+0000","last_clean":"2024-08-18T13:39:39.454861+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:39:39.454861+0000","last_undegraded":"2024-08-18T13:39:39.454861+0000","last_fullsized":"2024-08-18T13:39:39.454861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430338+0000","last_change":"2024-08-18T13:39:35.412156+0000","last_active":"2024-08-18T13:39:39.430338+0000","last_peered":"2024-08-18T13:39:39.430338+0000","last_clean":"2024-08-18T13:39:39.430338+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:39:39.430338+0000","last_undegraded":"2024-08-18T13:39:39.430338+0000","last_fullsized":"2024-08-18T13:39:39.430338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417339+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:39:39.417339+0000","last_peered":"2024-08-18T13:39:39.417339+0000","last_clean":"2024-08-18T13:39:39.417339+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:39:39.417339+0000","last_undegraded":"2024-08-18T13:39:39.417339+0000","last_fullsized":"2024-08-18T13:39:39.417339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450727+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:39:39.450727+0000","last_peered":"2024-08-18T13:39:39.450727+0000","last_clean":"2024-08-18T13:39:39.450727+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:39:39.450727+0000","last_undegraded":"2024-08-18T13:39:39.450727+0000","last_fullsized":"2024-08-18T13:39:39.450727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454332+0000","last_change":"2024-08-18T13:39:39.453360+0000","last_active":"2024-08-18T13:39:39.454332+0000","last_peered":"2024-08-18T13:39:39.454332+0000","last_clean":"2024-08-18T13:39:39.454332+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:39:39.454332+0000","last_undegraded":"2024-08-18T13:39:39.454332+0000","last_fullsized":"2024-08-18T13:39:39.454332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450556+0000","last_change":"2024-08-18T13:39:35.421765+0000","last_active":"2024-08-18T13:39:39.450556+0000","last_peered":"2024-08-18T13:39:39.450556+0000","last_clean":"2024-08-18T13:39:39.450556+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:39:39.450556+0000","last_undegraded":"2024-08-18T13:39:39.450556+0000","last_fullsized":"2024-08-18T13:39:39.450556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416404+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:39:39.416404+0000","last_peered":"2024-08-18T13:39:39.416404+0000","last_clean":"2024-08-18T13:39:39.416404+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:39:39.416404+0000","last_undegraded":"2024-08-18T13:39:39.416404+0000","last_fullsized":"2024-08-18T13:39:39.416404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450222+0000","last_change":"2024-08-18T13:39:39.449653+0000","last_active":"2024-08-18T13:39:39.450222+0000","last_peered":"2024-08-18T13:39:39.450222+0000","last_clean":"2024-08-18T13:39:39.450222+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:39:39.450222+0000","last_undegraded":"2024-08-18T13:39:39.450222+0000","last_fullsized":"2024-08-18T13:39:39.450222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422710+0000","last_change":"2024-08-18T13:39:35.407439+0000","last_active":"2024-08-18T13:39:39.422710+0000","last_peered":"2024-08-18T13:39:39.422710+0000","last_clean":"2024-08-18T13:39:39.422710+0000","last_became_active":"2024-08-18T13:39:35.407128+0000","last_became_peered":"2024-08-18T13:39:35.407128+0000","last_unstale":"2024-08-18T13:39:39.422710+0000","last_undegraded":"2024-08-18T13:39:39.422710+0000","last_fullsized":"2024-08-18T13:39:39.422710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431851+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:39:40.431851+0000","last_peered":"2024-08-18T13:39:40.431851+0000","last_clean":"2024-08-18T13:39:40.431851+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:39:40.431851+0000","last_undegraded":"2024-08-18T13:39:40.431851+0000","last_fullsized":"2024-08-18T13:39:40.431851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445844+0000","last_change":"2024-08-18T13:39:39.443848+0000","last_active":"2024-08-18T13:39:39.445844+0000","last_peered":"2024-08-18T13:39:39.445844+0000","last_clean":"2024-08-18T13:39:39.445844+0000","last_became_active":"2024-08-18T13:39:39.443456+0000","last_became_peered":"2024-08-18T13:39:39.443456+0000","last_unstale":"2024-08-18T13:39:39.445844+0000","last_undegraded":"2024-08-18T13:39:39.445844+0000","last_fullsized":"2024-08-18T13:39:39.445844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:27.994932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450757+0000","last_change":"2024-08-18T13:39:35.421612+0000","last_active":"2024-08-18T13:39:39.450757+0000","last_peered":"2024-08-18T13:39:39.450757+0000","last_clean":"2024-08-18T13:39:39.450757+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:39:39.450757+0000","last_undegraded":"2024-08-18T13:39:39.450757+0000","last_fullsized":"2024-08-18T13:39:39.450757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433442+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:39:40.433442+0000","last_peered":"2024-08-18T13:39:40.433442+0000","last_clean":"2024-08-18T13:39:40.433442+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:39:40.433442+0000","last_undegraded":"2024-08-18T13:39:40.433442+0000","last_fullsized":"2024-08-18T13:39:40.433442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454815+0000","last_change":"2024-08-18T13:39:39.453520+0000","last_active":"2024-08-18T13:39:39.454815+0000","last_peered":"2024-08-18T13:39:39.454815+0000","last_clean":"2024-08-18T13:39:39.454815+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:39:39.454815+0000","last_undegraded":"2024-08-18T13:39:39.454815+0000","last_fullsized":"2024-08-18T13:39:39.454815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450976+0000","last_change":"2024-08-18T13:39:35.421465+0000","last_active":"2024-08-18T13:39:39.450976+0000","last_peered":"2024-08-18T13:39:39.450976+0000","last_clean":"2024-08-18T13:39:39.450976+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:39:39.450976+0000","last_undegraded":"2024-08-18T13:39:39.450976+0000","last_fullsized":"2024-08-18T13:39:39.450976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429394+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:39:40.429394+0000","last_peered":"2024-08-18T13:39:40.429394+0000","last_clean":"2024-08-18T13:39:40.429394+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:39:40.429394+0000","last_undegraded":"2024-08-18T13:39:40.429394+0000","last_fullsized":"2024-08-18T13:39:40.429394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.448067+0000","last_change":"2024-08-18T13:39:39.446029+0000","last_active":"2024-08-18T13:39:39.448067+0000","last_peered":"2024-08-18T13:39:39.448067+0000","last_clean":"2024-08-18T13:39:39.448067+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:39:39.448067+0000","last_undegraded":"2024-08-18T13:39:39.448067+0000","last_fullsized":"2024-08-18T13:39:39.448067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450468+0000","last_change":"2024-08-18T13:39:35.417094+0000","last_active":"2024-08-18T13:39:39.450468+0000","last_peered":"2024-08-18T13:39:39.450468+0000","last_clean":"2024-08-18T13:39:39.450468+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:39:39.450468+0000","last_undegraded":"2024-08-18T13:39:39.450468+0000","last_fullsized":"2024-08-18T13:39:39.450468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.436721+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:39:40.436721+0000","last_peered":"2024-08-18T13:39:40.436721+0000","last_clean":"2024-08-18T13:39:40.436721+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:39:40.436721+0000","last_undegraded":"2024-08-18T13:39:40.436721+0000","last_fullsized":"2024-08-18T13:39:40.436721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437633+0000","last_change":"2024-08-18T13:39:39.436992+0000","last_active":"2024-08-18T13:39:39.437633+0000","last_peered":"2024-08-18T13:39:39.437633+0000","last_clean":"2024-08-18T13:39:39.437633+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:39:39.437633+0000","last_undegraded":"2024-08-18T13:39:39.437633+0000","last_fullsized":"2024-08-18T13:39:39.437633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427745+0000","last_change":"2024-08-18T13:39:35.411315+0000","last_active":"2024-08-18T13:39:39.427745+0000","last_peered":"2024-08-18T13:39:39.427745+0000","last_clean":"2024-08-18T13:39:39.427745+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:39:39.427745+0000","last_undegraded":"2024-08-18T13:39:39.427745+0000","last_fullsized":"2024-08-18T13:39:39.427745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428272+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:39:39.428272+0000","last_peered":"2024-08-18T13:39:39.428272+0000","last_clean":"2024-08-18T13:39:39.428272+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:39:39.428272+0000","last_undegraded":"2024-08-18T13:39:39.428272+0000","last_fullsized":"2024-08-18T13:39:39.428272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.458299+0000","last_change":"2024-08-18T13:39:39.456955+0000","last_active":"2024-08-18T13:39:39.458299+0000","last_peered":"2024-08-18T13:39:39.458299+0000","last_clean":"2024-08-18T13:39:39.458299+0000","last_became_active":"2024-08-18T13:39:39.456534+0000","last_became_peered":"2024-08-18T13:39:39.456534+0000","last_unstale":"2024-08-18T13:39:39.458299+0000","last_undegraded":"2024-08-18T13:39:39.458299+0000","last_fullsized":"2024-08-18T13:39:39.458299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:26:42.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450840+0000","last_change":"2024-08-18T13:39:35.417348+0000","last_active":"2024-08-18T13:39:39.450840+0000","last_peered":"2024-08-18T13:39:39.450840+0000","last_clean":"2024-08-18T13:39:39.450840+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:39:39.450840+0000","last_undegraded":"2024-08-18T13:39:39.450840+0000","last_fullsized":"2024-08-18T13:39:39.450840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417725+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:39:39.417725+0000","last_peered":"2024-08-18T13:39:39.417725+0000","last_clean":"2024-08-18T13:39:39.417725+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:39:39.417725+0000","last_undegraded":"2024-08-18T13:39:39.417725+0000","last_fullsized":"2024-08-18T13:39:39.417725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454724+0000","last_change":"2024-08-18T13:39:39.453682+0000","last_active":"2024-08-18T13:39:39.454724+0000","last_peered":"2024-08-18T13:39:39.454724+0000","last_clean":"2024-08-18T13:39:39.454724+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:39:39.454724+0000","last_undegraded":"2024-08-18T13:39:39.454724+0000","last_fullsized":"2024-08-18T13:39:39.454724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427706+0000","last_change":"2024-08-18T13:39:35.407127+0000","last_active":"2024-08-18T13:39:39.427706+0000","last_peered":"2024-08-18T13:39:39.427706+0000","last_clean":"2024-08-18T13:39:39.427706+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:39:39.427706+0000","last_undegraded":"2024-08-18T13:39:39.427706+0000","last_fullsized":"2024-08-18T13:39:39.427706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431523+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:39:40.431523+0000","last_peered":"2024-08-18T13:39:40.431523+0000","last_clean":"2024-08-18T13:39:40.431523+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:39:40.431523+0000","last_undegraded":"2024-08-18T13:39:40.431523+0000","last_fullsized":"2024-08-18T13:39:40.431523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457313+0000","last_change":"2024-08-18T13:39:39.456626+0000","last_active":"2024-08-18T13:39:39.457313+0000","last_peered":"2024-08-18T13:39:39.457313+0000","last_clean":"2024-08-18T13:39:39.457313+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:39:39.457313+0000","last_undegraded":"2024-08-18T13:39:39.457313+0000","last_fullsized":"2024-08-18T13:39:39.457313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450395+0000","last_change":"2024-08-18T13:39:35.417693+0000","last_active":"2024-08-18T13:39:39.450395+0000","last_peered":"2024-08-18T13:39:39.450395+0000","last_clean":"2024-08-18T13:39:39.450395+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:39:39.450395+0000","last_undegraded":"2024-08-18T13:39:39.450395+0000","last_fullsized":"2024-08-18T13:39:39.450395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:40:09.449886+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:40:09.449886+0000","last_peered":"2024-08-18T13:40:09.449886+0000","last_clean":"2024-08-18T13:40:09.449886+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:40:09.449886+0000","last_undegraded":"2024-08-18T13:40:09.449886+0000","last_fullsized":"2024-08-18T13:40:09.449886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4207,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.440639+0000","last_change":"2024-08-18T13:39:39.438064+0000","last_active":"2024-08-18T13:39:39.440639+0000","last_peered":"2024-08-18T13:39:39.440639+0000","last_clean":"2024-08-18T13:39:39.440639+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:39:39.440639+0000","last_undegraded":"2024-08-18T13:39:39.440639+0000","last_fullsized":"2024-08-18T13:39:39.440639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421839+0000","last_change":"2024-08-18T13:39:35.406398+0000","last_active":"2024-08-18T13:39:39.421839+0000","last_peered":"2024-08-18T13:39:39.421839+0000","last_clean":"2024-08-18T13:39:39.421839+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:39:39.421839+0000","last_undegraded":"2024-08-18T13:39:39.421839+0000","last_fullsized":"2024-08-18T13:39:39.421839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433257+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:39:40.433257+0000","last_peered":"2024-08-18T13:39:40.433257+0000","last_clean":"2024-08-18T13:39:40.433257+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:39:40.433257+0000","last_undegraded":"2024-08-18T13:39:40.433257+0000","last_fullsized":"2024-08-18T13:39:40.433257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444385+0000","last_change":"2024-08-18T13:39:39.439962+0000","last_active":"2024-08-18T13:39:39.444385+0000","last_peered":"2024-08-18T13:39:39.444385+0000","last_clean":"2024-08-18T13:39:39.444385+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:39:39.444385+0000","last_undegraded":"2024-08-18T13:39:39.444385+0000","last_fullsized":"2024-08-18T13:39:39.444385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432863+0000","last_change":"2024-08-18T13:39:35.411067+0000","last_active":"2024-08-18T13:39:39.432863+0000","last_peered":"2024-08-18T13:39:39.432863+0000","last_clean":"2024-08-18T13:39:39.432863+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:39:39.432863+0000","last_undegraded":"2024-08-18T13:39:39.432863+0000","last_fullsized":"2024-08-18T13:39:39.432863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422811+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:39:39.422811+0000","last_peered":"2024-08-18T13:39:39.422811+0000","last_clean":"2024-08-18T13:39:39.422811+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:39:39.422811+0000","last_undegraded":"2024-08-18T13:39:39.422811+0000","last_fullsized":"2024-08-18T13:39:39.422811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447069+0000","last_change":"2024-08-18T13:39:39.443938+0000","last_active":"2024-08-18T13:39:39.447069+0000","last_peered":"2024-08-18T13:39:39.447069+0000","last_clean":"2024-08-18T13:39:39.447069+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:39:39.447069+0000","last_undegraded":"2024-08-18T13:39:39.447069+0000","last_fullsized":"2024-08-18T13:39:39.447069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422769+0000","last_change":"2024-08-18T13:39:35.409517+0000","last_active":"2024-08-18T13:39:39.422769+0000","last_peered":"2024-08-18T13:39:39.422769+0000","last_clean":"2024-08-18T13:39:39.422769+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:39:39.422769+0000","last_undegraded":"2024-08-18T13:39:39.422769+0000","last_fullsized":"2024-08-18T13:39:39.422769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429841+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:39:39.429841+0000","last_peered":"2024-08-18T13:39:39.429841+0000","last_clean":"2024-08-18T13:39:39.429841+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:39:39.429841+0000","last_undegraded":"2024-08-18T13:39:39.429841+0000","last_fullsized":"2024-08-18T13:39:39.429841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450637+0000","last_change":"2024-08-18T13:39:39.449671+0000","last_active":"2024-08-18T13:39:39.450637+0000","last_peered":"2024-08-18T13:39:39.450637+0000","last_clean":"2024-08-18T13:39:39.450637+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:39:39.450637+0000","last_undegraded":"2024-08-18T13:39:39.450637+0000","last_fullsized":"2024-08-18T13:39:39.450637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454824+0000","last_change":"2024-08-18T13:39:39.453907+0000","last_active":"2024-08-18T13:39:39.454824+0000","last_peered":"2024-08-18T13:39:39.454824+0000","last_clean":"2024-08-18T13:39:39.454824+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:39:39.454824+0000","last_undegraded":"2024-08-18T13:39:39.454824+0000","last_fullsized":"2024-08-18T13:39:39.454824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422264+0000","last_change":"2024-08-18T13:39:35.401997+0000","last_active":"2024-08-18T13:39:39.422264+0000","last_peered":"2024-08-18T13:39:39.422264+0000","last_clean":"2024-08-18T13:39:39.422264+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:39:39.422264+0000","last_undegraded":"2024-08-18T13:39:39.422264+0000","last_fullsized":"2024-08-18T13:39:39.422264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418448+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:39:39.418448+0000","last_peered":"2024-08-18T13:39:39.418448+0000","last_clean":"2024-08-18T13:39:39.418448+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:39:39.418448+0000","last_undegraded":"2024-08-18T13:39:39.418448+0000","last_fullsized":"2024-08-18T13:39:39.418448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442977+0000","last_change":"2024-08-18T13:39:39.440811+0000","last_active":"2024-08-18T13:39:39.442977+0000","last_peered":"2024-08-18T13:39:39.442977+0000","last_clean":"2024-08-18T13:39:39.442977+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:39:39.442977+0000","last_undegraded":"2024-08-18T13:39:39.442977+0000","last_fullsized":"2024-08-18T13:39:39.442977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430210+0000","last_change":"2024-08-18T13:39:35.412127+0000","last_active":"2024-08-18T13:39:39.430210+0000","last_peered":"2024-08-18T13:39:39.430210+0000","last_clean":"2024-08-18T13:39:39.430210+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:39:39.430210+0000","last_undegraded":"2024-08-18T13:39:39.430210+0000","last_fullsized":"2024-08-18T13:39:39.430210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423847+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:39:39.423847+0000","last_peered":"2024-08-18T13:39:39.423847+0000","last_clean":"2024-08-18T13:39:39.423847+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:39:39.423847+0000","last_undegraded":"2024-08-18T13:39:39.423847+0000","last_fullsized":"2024-08-18T13:39:39.423847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429678+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:39:40.429678+0000","last_peered":"2024-08-18T13:39:40.429678+0000","last_clean":"2024-08-18T13:39:40.429678+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:39:40.429678+0000","last_undegraded":"2024-08-18T13:39:40.429678+0000","last_fullsized":"2024-08-18T13:39:40.429678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429574+0000","last_change":"2024-08-18T13:39:35.407995+0000","last_active":"2024-08-18T13:39:39.429574+0000","last_peered":"2024-08-18T13:39:39.429574+0000","last_clean":"2024-08-18T13:39:39.429574+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:39:39.429574+0000","last_undegraded":"2024-08-18T13:39:39.429574+0000","last_fullsized":"2024-08-18T13:39:39.429574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.438294+0000","last_change":"2024-08-18T13:39:39.437542+0000","last_active":"2024-08-18T13:39:39.438294+0000","last_peered":"2024-08-18T13:39:39.438294+0000","last_clean":"2024-08-18T13:39:39.438294+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:39:39.438294+0000","last_undegraded":"2024-08-18T13:39:39.438294+0000","last_fullsized":"2024-08-18T13:39:39.438294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434310+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:39:40.434310+0000","last_peered":"2024-08-18T13:39:40.434310+0000","last_clean":"2024-08-18T13:39:40.434310+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:39:40.434310+0000","last_undegraded":"2024-08-18T13:39:40.434310+0000","last_fullsized":"2024-08-18T13:39:40.434310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422760+0000","last_change":"2024-08-18T13:39:35.403759+0000","last_active":"2024-08-18T13:39:39.422760+0000","last_peered":"2024-08-18T13:39:39.422760+0000","last_clean":"2024-08-18T13:39:39.422760+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:39:39.422760+0000","last_undegraded":"2024-08-18T13:39:39.422760+0000","last_fullsized":"2024-08-18T13:39:39.422760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446432+0000","last_change":"2024-08-18T13:39:39.441201+0000","last_active":"2024-08-18T13:39:39.446432+0000","last_peered":"2024-08-18T13:39:39.446432+0000","last_clean":"2024-08-18T13:39:39.446432+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:39:39.446432+0000","last_undegraded":"2024-08-18T13:39:39.446432+0000","last_fullsized":"2024-08-18T13:39:39.446432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427616+0000","last_change":"2024-08-18T13:39:35.413292+0000","last_active":"2024-08-18T13:39:39.427616+0000","last_peered":"2024-08-18T13:39:39.427616+0000","last_clean":"2024-08-18T13:39:39.427616+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:39:39.427616+0000","last_undegraded":"2024-08-18T13:39:39.427616+0000","last_fullsized":"2024-08-18T13:39:39.427616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415812+0000","last_change":"2024-08-18T13:39:35.401196+0000","last_active":"2024-08-18T13:39:39.415812+0000","last_peered":"2024-08-18T13:39:39.415812+0000","last_clean":"2024-08-18T13:39:39.415812+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:39:39.415812+0000","last_undegraded":"2024-08-18T13:39:39.415812+0000","last_fullsized":"2024-08-18T13:39:39.415812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417577+0000","last_change":"2024-08-18T13:39:35.406603+0000","last_active":"2024-08-18T13:39:39.417577+0000","last_peered":"2024-08-18T13:39:39.417577+0000","last_clean":"2024-08-18T13:39:39.417577+0000","last_became_active":"2024-08-18T13:39:35.406331+0000","last_became_peered":"2024-08-18T13:39:35.406331+0000","last_unstale":"2024-08-18T13:39:39.417577+0000","last_undegraded":"2024-08-18T13:39:39.417577+0000","last_fullsized":"2024-08-18T13:39:39.417577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:21:12.232172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429494+0000","last_change":"2024-08-18T13:39:35.404051+0000","last_active":"2024-08-18T13:39:39.429494+0000","last_peered":"2024-08-18T13:39:39.429494+0000","last_clean":"2024-08-18T13:39:39.429494+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:39:39.429494+0000","last_undegraded":"2024-08-18T13:39:39.429494+0000","last_fullsized":"2024-08-18T13:39:39.429494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422959+0000","last_change":"2024-08-18T13:39:35.409288+0000","last_active":"2024-08-18T13:39:39.422959+0000","last_peered":"2024-08-18T13:39:39.422959+0000","last_clean":"2024-08-18T13:39:39.422959+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:39:39.422959+0000","last_undegraded":"2024-08-18T13:39:39.422959+0000","last_fullsized":"2024-08-18T13:39:39.422959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424121+0000","last_change":"2024-08-18T13:39:35.403051+0000","last_active":"2024-08-18T13:39:39.424121+0000","last_peered":"2024-08-18T13:39:39.424121+0000","last_clean":"2024-08-18T13:39:39.424121+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:39:39.424121+0000","last_undegraded":"2024-08-18T13:39:39.424121+0000","last_fullsized":"2024-08-18T13:39:39.424121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430049+0000","last_change":"2024-08-18T13:39:35.412281+0000","last_active":"2024-08-18T13:39:39.430049+0000","last_peered":"2024-08-18T13:39:39.430049+0000","last_clean":"2024-08-18T13:39:39.430049+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:39:39.430049+0000","last_undegraded":"2024-08-18T13:39:39.430049+0000","last_fullsized":"2024-08-18T13:39:39.430049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424466+0000","last_change":"2024-08-18T13:39:35.409220+0000","last_active":"2024-08-18T13:39:39.424466+0000","last_peered":"2024-08-18T13:39:39.424466+0000","last_clean":"2024-08-18T13:39:39.424466+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:39:39.424466+0000","last_undegraded":"2024-08-18T13:39:39.424466+0000","last_fullsized":"2024-08-18T13:39:39.424466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418253+0000","last_change":"2024-08-18T13:39:35.401759+0000","last_active":"2024-08-18T13:39:39.418253+0000","last_peered":"2024-08-18T13:39:39.418253+0000","last_clean":"2024-08-18T13:39:39.418253+0000","last_became_active":"2024-08-18T13:39:35.401476+0000","last_became_peered":"2024-08-18T13:39:35.401476+0000","last_unstale":"2024-08-18T13:39:39.418253+0000","last_undegraded":"2024-08-18T13:39:39.418253+0000","last_fullsized":"2024-08-18T13:39:39.418253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:04:43.997597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423669+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.423669+0000","last_peered":"2024-08-18T13:39:39.423669+0000","last_clean":"2024-08-18T13:39:39.423669+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:39:39.423669+0000","last_undegraded":"2024-08-18T13:39:39.423669+0000","last_fullsized":"2024-08-18T13:39:39.423669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417410+0000","last_change":"2024-08-18T13:39:35.407218+0000","last_active":"2024-08-18T13:39:39.417410+0000","last_peered":"2024-08-18T13:39:39.417410+0000","last_clean":"2024-08-18T13:39:39.417410+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:39:39.417410+0000","last_undegraded":"2024-08-18T13:39:39.417410+0000","last_fullsized":"2024-08-18T13:39:39.417410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429387+0000","last_change":"2024-08-18T13:39:35.407339+0000","last_active":"2024-08-18T13:39:39.429387+0000","last_peered":"2024-08-18T13:39:39.429387+0000","last_clean":"2024-08-18T13:39:39.429387+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:39:39.429387+0000","last_undegraded":"2024-08-18T13:39:39.429387+0000","last_fullsized":"2024-08-18T13:39:39.429387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429062+0000","last_change":"2024-08-18T13:39:35.408647+0000","last_active":"2024-08-18T13:39:39.429062+0000","last_peered":"2024-08-18T13:39:39.429062+0000","last_clean":"2024-08-18T13:39:39.429062+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:39:39.429062+0000","last_undegraded":"2024-08-18T13:39:39.429062+0000","last_fullsized":"2024-08-18T13:39:39.429062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450837+0000","last_change":"2024-08-18T13:39:35.418000+0000","last_active":"2024-08-18T13:39:39.450837+0000","last_peered":"2024-08-18T13:39:39.450837+0000","last_clean":"2024-08-18T13:39:39.450837+0000","last_became_active":"2024-08-18T13:39:35.417661+0000","last_became_peered":"2024-08-18T13:39:35.417661+0000","last_unstale":"2024-08-18T13:39:39.450837+0000","last_undegraded":"2024-08-18T13:39:39.450837+0000","last_fullsized":"2024-08-18T13:39:39.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:53:44.739585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416124+0000","last_change":"2024-08-18T13:39:35.405425+0000","last_active":"2024-08-18T13:39:39.416124+0000","last_peered":"2024-08-18T13:39:39.416124+0000","last_clean":"2024-08-18T13:39:39.416124+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:39:39.416124+0000","last_undegraded":"2024-08-18T13:39:39.416124+0000","last_fullsized":"2024-08-18T13:39:39.416124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423886+0000","last_change":"2024-08-18T13:39:35.400889+0000","last_active":"2024-08-18T13:39:39.423886+0000","last_peered":"2024-08-18T13:39:39.423886+0000","last_clean":"2024-08-18T13:39:39.423886+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:39:39.423886+0000","last_undegraded":"2024-08-18T13:39:39.423886+0000","last_fullsized":"2024-08-18T13:39:39.423886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418057+0000","last_change":"2024-08-18T13:39:35.407634+0000","last_active":"2024-08-18T13:39:39.418057+0000","last_peered":"2024-08-18T13:39:39.418057+0000","last_clean":"2024-08-18T13:39:39.418057+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:39:39.418057+0000","last_undegraded":"2024-08-18T13:39:39.418057+0000","last_fullsized":"2024-08-18T13:39:39.418057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422473+0000","last_change":"2024-08-18T13:39:35.397509+0000","last_active":"2024-08-18T13:39:39.422473+0000","last_peered":"2024-08-18T13:39:39.422473+0000","last_clean":"2024-08-18T13:39:39.422473+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:39:39.422473+0000","last_undegraded":"2024-08-18T13:39:39.422473+0000","last_fullsized":"2024-08-18T13:39:39.422473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425330+0000","last_change":"2024-08-18T13:39:35.404533+0000","last_active":"2024-08-18T13:39:39.425330+0000","last_peered":"2024-08-18T13:39:39.425330+0000","last_clean":"2024-08-18T13:39:39.425330+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:39:39.425330+0000","last_undegraded":"2024-08-18T13:39:39.425330+0000","last_fullsized":"2024-08-18T13:39:39.425330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425567+0000","last_change":"2024-08-18T13:39:35.404839+0000","last_active":"2024-08-18T13:39:39.425567+0000","last_peered":"2024-08-18T13:39:39.425567+0000","last_clean":"2024-08-18T13:39:39.425567+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:39:39.425567+0000","last_undegraded":"2024-08-18T13:39:39.425567+0000","last_fullsized":"2024-08-18T13:39:39.425567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421633+0000","last_change":"2024-08-18T13:39:35.405867+0000","last_active":"2024-08-18T13:39:39.421633+0000","last_peered":"2024-08-18T13:39:39.421633+0000","last_clean":"2024-08-18T13:39:39.421633+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:39:39.421633+0000","last_undegraded":"2024-08-18T13:39:39.421633+0000","last_fullsized":"2024-08-18T13:39:39.421633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429697+0000","last_change":"2024-08-18T13:39:35.411173+0000","last_active":"2024-08-18T13:39:39.429697+0000","last_peered":"2024-08-18T13:39:39.429697+0000","last_clean":"2024-08-18T13:39:39.429697+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:39:39.429697+0000","last_undegraded":"2024-08-18T13:39:39.429697+0000","last_fullsized":"2024-08-18T13:39:39.429697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429962+0000","last_change":"2024-08-18T13:39:35.418013+0000","last_active":"2024-08-18T13:39:39.429962+0000","last_peered":"2024-08-18T13:39:39.429962+0000","last_clean":"2024-08-18T13:39:39.429962+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:39:39.429962+0000","last_undegraded":"2024-08-18T13:39:39.429962+0000","last_fullsized":"2024-08-18T13:39:39.429962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450803+0000","last_change":"2024-08-18T13:39:35.421633+0000","last_active":"2024-08-18T13:39:39.450803+0000","last_peered":"2024-08-18T13:39:39.450803+0000","last_clean":"2024-08-18T13:39:39.450803+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:39:39.450803+0000","last_undegraded":"2024-08-18T13:39:39.450803+0000","last_fullsized":"2024-08-18T13:39:39.450803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425234+0000","last_change":"2024-08-18T13:39:35.411443+0000","last_active":"2024-08-18T13:39:39.425234+0000","last_peered":"2024-08-18T13:39:39.425234+0000","last_clean":"2024-08-18T13:39:39.425234+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:39:39.425234+0000","last_undegraded":"2024-08-18T13:39:39.425234+0000","last_fullsized":"2024-08-18T13:39:39.425234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450366+0000","last_change":"2024-08-18T13:39:35.415548+0000","last_active":"2024-08-18T13:39:39.450366+0000","last_peered":"2024-08-18T13:39:39.450366+0000","last_clean":"2024-08-18T13:39:39.450366+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:39:39.450366+0000","last_undegraded":"2024-08-18T13:39:39.450366+0000","last_fullsized":"2024-08-18T13:39:39.450366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417246+0000","last_change":"2024-08-18T13:39:35.410795+0000","last_active":"2024-08-18T13:39:39.417246+0000","last_peered":"2024-08-18T13:39:39.417246+0000","last_clean":"2024-08-18T13:39:39.417246+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:39:39.417246+0000","last_undegraded":"2024-08-18T13:39:39.417246+0000","last_fullsized":"2024-08-18T13:39:39.417246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450710+0000","last_change":"2024-08-18T13:39:35.417550+0000","last_active":"2024-08-18T13:39:39.450710+0000","last_peered":"2024-08-18T13:39:39.450710+0000","last_clean":"2024-08-18T13:39:39.450710+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:39:39.450710+0000","last_undegraded":"2024-08-18T13:39:39.450710+0000","last_fullsized":"2024-08-18T13:39:39.450710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422555+0000","last_change":"2024-08-18T13:39:35.410002+0000","last_active":"2024-08-18T13:39:39.422555+0000","last_peered":"2024-08-18T13:39:39.422555+0000","last_clean":"2024-08-18T13:39:39.422555+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:39:39.422555+0000","last_undegraded":"2024-08-18T13:39:39.422555+0000","last_fullsized":"2024-08-18T13:39:39.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427751+0000","last_change":"2024-08-18T13:39:35.406627+0000","last_active":"2024-08-18T13:39:39.427751+0000","last_peered":"2024-08-18T13:39:39.427751+0000","last_clean":"2024-08-18T13:39:39.427751+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:39:39.427751+0000","last_undegraded":"2024-08-18T13:39:39.427751+0000","last_fullsized":"2024-08-18T13:39:39.427751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423078+0000","last_change":"2024-08-18T13:39:35.405750+0000","last_active":"2024-08-18T13:39:39.423078+0000","last_peered":"2024-08-18T13:39:39.423078+0000","last_clean":"2024-08-18T13:39:39.423078+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:39:39.423078+0000","last_undegraded":"2024-08-18T13:39:39.423078+0000","last_fullsized":"2024-08-18T13:39:39.423078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450591+0000","last_change":"2024-08-18T13:39:35.417195+0000","last_active":"2024-08-18T13:39:39.450591+0000","last_peered":"2024-08-18T13:39:39.450591+0000","last_clean":"2024-08-18T13:39:39.450591+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:39:39.450591+0000","last_undegraded":"2024-08-18T13:39:39.450591+0000","last_fullsized":"2024-08-18T13:39:39.450591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4993,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":856064,"data_stored":693001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":270336,"data_stored":102189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":258048,"data_stored":102063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]}]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":102223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":864256,"data_stored":693057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":262144,"data_stored":102153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:40:15.124 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-08-18T13:40:15.125 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:40:15.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66dc3b9700 1 Processor -- start 2024-08-18T13:40:15.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66dc3b9700 1 -- start start 2024-08-18T13:40:15.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66dc3b9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f66d414bb10 0x7f66d40c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66dc3b9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66d40c56a0 0x7f66d40c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66dc3b9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d40673d0 con 0x7f66d414b740 2024-08-18T13:40:15.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66dc3b9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d4151830 con 0x7f66d414bb10 2024-08-18T13:40:15.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66dc3b9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d4151ab0 con 0x7f66d40c56a0 2024-08-18T13:40:15.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66d9954700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f66d414bb10 0x7f66d40c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66da155700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f66d414b740 0x7f66d4151f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53932/0 (socket says 172.21.15.118:53932) 2024-08-18T13:40:15.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.222+0000 7f66da155700 1 -- 172.21.15.118:0/2855273788 learned_addr learned my addr 172.21.15.118:0/2855273788 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66da956700 1 --2- 172.21.15.118:0/2855273788 >> v2:172.21.15.178:3300/0 conn(0x7f66d40c56a0 0x7f66d40c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9954700 1 -- 172.21.15.118:0/2855273788 >> v2:172.21.15.178:3300/0 conn(0x7f66d40c56a0 msgr2=0x7f66d40c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9954700 1 --2- 172.21.15.118:0/2855273788 >> v2:172.21.15.178:3300/0 conn(0x7f66d40c56a0 0x7f66d40c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9153700 1 -- 172.21.15.118:0/2855273788 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830171128 0 0) 0x7f66d40673d0 con 0x7f66d414b740 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9954700 1 -- 172.21.15.118:0/2855273788 >> v1:172.21.15.118:6789/0 conn(0x7f66d414b740 legacy=0x7f66d4151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9954700 1 -- 172.21.15.118:0/2855273788 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d40c2e80 con 0x7f66d414bb10 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9954700 1 --2- 172.21.15.118:0/2855273788 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d414bb10 0x7f66d40c37c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66d40c5f80 tx=0x7f66c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf74ef178e62976 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9153700 1 -- 172.21.15.118:0/2855273788 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400c7c0 con 0x7f66d414bb10 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9153700 1 -- 172.21.15.118:0/2855273788 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c400c920 con 0x7f66d414bb10 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66d9153700 1 -- 172.21.15.118:0/2855273788 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400ccc0 con 0x7f66d414bb10 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 -- 172.21.15.118:0/2855273788 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d414bb10 msgr2=0x7f66d40c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/2855273788 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d414bb10 0x7f66d40c37c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66d40c5f80 tx=0x7f66c40091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.231 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 -- 172.21.15.118:0/2855273788 shutdown_connections 2024-08-18T13:40:15.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/2855273788 >> v2:172.21.15.178:3300/0 conn(0x7f66d40c56a0 0x7f66d40c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/2855273788 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d414bb10 0x7f66d40c37c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 -- 172.21.15.118:0/2855273788 >> 172.21.15.118:0/2855273788 conn(0x7f66d40c8e70 msgr2=0x7f66d40c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:15.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 -- 172.21.15.118:0/2855273788 shutdown_connections 2024-08-18T13:40:15.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 -- 172.21.15.118:0/2855273788 wait complete. 2024-08-18T13:40:15.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 Processor -- start 2024-08-18T13:40:15.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.226+0000 7f66dc3b9700 1 -- start start 2024-08-18T13:40:15.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d40c56a0 0x7f66d406b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66d414b740 0x7f66d40c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66d414bb10 0x7f66d413bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4154560 con 0x7f66d40c56a0 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d40c1750 con 0x7f66d414b740 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d41547b0 con 0x7f66d414bb10 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d40c56a0 0x7f66d406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d40c56a0 0x7f66d406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51978/0 (socket says 172.21.15.118:51978) 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 -- 172.21.15.118:0/435280269 learned_addr learned my addr 172.21.15.118:0/435280269 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66d9954700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66d414b740 0x7f66d40c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da956700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66d414bb10 0x7f66d413bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 -- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66d414bb10 msgr2=0x7f66d413bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66d414bb10 0x7f66d413bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 -- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66d414b740 msgr2=0x7f66d40c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66d414b740 0x7f66d40c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 -- 172.21.15.118:0/435280269 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4009420 con 0x7f66d40c56a0 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da956700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66d414bb10 0x7f66d413bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66da155700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d40c56a0 0x7f66d406b900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c7e0 tx=0x7f66d0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66ca7fc700 1 -- 172.21.15.118:0/435280269 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0005b30 con 0x7f66d40c56a0 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66ca7fc700 1 -- 172.21.15.118:0/435280269 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d0005c90 con 0x7f66d40c56a0 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d4154a90 con 0x7f66d40c56a0 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66ca7fc700 1 -- 172.21.15.118:0/435280269 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0007300 con 0x7f66d40c56a0 2024-08-18T13:40:15.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d40c0a80 con 0x7f66d40c56a0 2024-08-18T13:40:15.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d40648f0 con 0x7f66d40c56a0 2024-08-18T13:40:15.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66ca7fc700 1 -- 172.21.15.118:0/435280269 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f66d001b400 con 0x7f66d40c56a0 2024-08-18T13:40:15.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.230+0000 7f66ca7fc700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060890 0x7f66b8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.234+0000 7f66ca7fc700 1 -- 172.21.15.118:0/435280269 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f66d007ca40 con 0x7f66d40c56a0 2024-08-18T13:40:15.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.234+0000 7f66d9954700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060890 0x7f66b8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.234+0000 7f66d9954700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060890 0x7f66b8062d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66d40c5f80 tx=0x7f66c4018000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.238+0000 7f66ca7fc700 1 -- 172.21.15.118:0/435280269 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f66d007ee60 con 0x7f66d40c56a0 2024-08-18T13:40:15.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.402+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66d406dd70 con 0x7f66b8060890 2024-08-18T13:40:15.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.414+0000 7f66ca7fc700 1 -- 172.21.15.118:0/435280269 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345233 (secure 0 0 0) 0x7f66d406dd70 con 0x7f66b8060890 2024-08-18T13:40:15.424 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060890 msgr2=0x7f66b8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060890 0x7f66b8062d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66d40c5f80 tx=0x7f66c4018000 comp rx=0 tx=0).stop 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d40c56a0 msgr2=0x7f66d406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d40c56a0 0x7f66d406b900 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c7e0 tx=0x7f66d0007df0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 shutdown_connections 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060890 0x7f66b8062d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66d414bb10 0x7f66d413bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66d414b740 0x7f66d40c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 --2- 172.21.15.118:0/435280269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66d40c56a0 0x7f66d406b900 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 >> 172.21.15.118:0/435280269 conn(0x7f66d40c8e70 msgr2=0x7f66d4150220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 shutdown_connections 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.422+0000 7f66dc3b9700 1 -- 172.21.15.118:0/435280269 wait complete. 2024-08-18T13:40:15.427 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:40:15.439 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-08-18T13:40:15.211851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595361,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":209,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2415368,"kb_used_data":3240,"kb_used_omap":13,"kb_used_meta":2412018,"kb_avail":752559352,"statfs":{"total":773094113280,"available":770620776448,"internally_reserved":0,"allocated":3317760,"data_stored":2007226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2469907136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":445,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005582"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433629+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:39:40.433629+0000","last_peered":"2024-08-18T13:39:40.433629+0000","last_clean":"2024-08-18T13:39:40.433629+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:39:40.433629+0000","last_undegraded":"2024-08-18T13:39:40.433629+0000","last_fullsized":"2024-08-18T13:39:40.433629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422872+0000","last_change":"2024-08-18T13:39:35.405966+0000","last_active":"2024-08-18T13:39:39.422872+0000","last_peered":"2024-08-18T13:39:39.422872+0000","last_clean":"2024-08-18T13:39:39.422872+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:39:39.422872+0000","last_undegraded":"2024-08-18T13:39:39.422872+0000","last_fullsized":"2024-08-18T13:39:39.422872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454715+0000","last_change":"2024-08-18T13:39:39.453674+0000","last_active":"2024-08-18T13:39:39.454715+0000","last_peered":"2024-08-18T13:39:39.454715+0000","last_clean":"2024-08-18T13:39:39.454715+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:39:39.454715+0000","last_undegraded":"2024-08-18T13:39:39.454715+0000","last_fullsized":"2024-08-18T13:39:39.454715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434410+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:39:40.434410+0000","last_peered":"2024-08-18T13:39:40.434410+0000","last_clean":"2024-08-18T13:39:40.434410+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:39:40.434410+0000","last_undegraded":"2024-08-18T13:39:40.434410+0000","last_fullsized":"2024-08-18T13:39:40.434410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432862+0000","last_change":"2024-08-18T13:39:35.408917+0000","last_active":"2024-08-18T13:39:39.432862+0000","last_peered":"2024-08-18T13:39:39.432862+0000","last_clean":"2024-08-18T13:39:39.432862+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:39:39.432862+0000","last_undegraded":"2024-08-18T13:39:39.432862+0000","last_fullsized":"2024-08-18T13:39:39.432862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447766+0000","last_change":"2024-08-18T13:39:39.445388+0000","last_active":"2024-08-18T13:39:39.447766+0000","last_peered":"2024-08-18T13:39:39.447766+0000","last_clean":"2024-08-18T13:39:39.447766+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:39:39.447766+0000","last_undegraded":"2024-08-18T13:39:39.447766+0000","last_fullsized":"2024-08-18T13:39:39.447766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430049+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:39:40.430049+0000","last_peered":"2024-08-18T13:39:40.430049+0000","last_clean":"2024-08-18T13:39:40.430049+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:39:40.430049+0000","last_undegraded":"2024-08-18T13:39:40.430049+0000","last_fullsized":"2024-08-18T13:39:40.430049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422649+0000","last_change":"2024-08-18T13:39:35.399188+0000","last_active":"2024-08-18T13:39:39.422649+0000","last_peered":"2024-08-18T13:39:39.422649+0000","last_clean":"2024-08-18T13:39:39.422649+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:39:39.422649+0000","last_undegraded":"2024-08-18T13:39:39.422649+0000","last_fullsized":"2024-08-18T13:39:39.422649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447285+0000","last_change":"2024-08-18T13:39:39.440922+0000","last_active":"2024-08-18T13:39:39.447285+0000","last_peered":"2024-08-18T13:39:39.447285+0000","last_clean":"2024-08-18T13:39:39.447285+0000","last_became_active":"2024-08-18T13:39:39.440426+0000","last_became_peered":"2024-08-18T13:39:39.440426+0000","last_unstale":"2024-08-18T13:39:39.447285+0000","last_undegraded":"2024-08-18T13:39:39.447285+0000","last_fullsized":"2024-08-18T13:39:39.447285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:23:00.376805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.428764+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:39:40.428764+0000","last_peered":"2024-08-18T13:39:40.428764+0000","last_clean":"2024-08-18T13:39:40.428764+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:39:40.428764+0000","last_undegraded":"2024-08-18T13:39:40.428764+0000","last_fullsized":"2024-08-18T13:39:40.428764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423184+0000","last_change":"2024-08-18T13:39:35.400038+0000","last_active":"2024-08-18T13:39:39.423184+0000","last_peered":"2024-08-18T13:39:39.423184+0000","last_clean":"2024-08-18T13:39:39.423184+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:39:39.423184+0000","last_undegraded":"2024-08-18T13:39:39.423184+0000","last_fullsized":"2024-08-18T13:39:39.423184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437377+0000","last_change":"2024-08-18T13:39:39.436221+0000","last_active":"2024-08-18T13:39:39.437377+0000","last_peered":"2024-08-18T13:39:39.437377+0000","last_clean":"2024-08-18T13:39:39.437377+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:39:39.437377+0000","last_undegraded":"2024-08-18T13:39:39.437377+0000","last_fullsized":"2024-08-18T13:39:39.437377+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428676+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.428676+0000","last_peered":"2024-08-18T13:39:39.428676+0000","last_clean":"2024-08-18T13:39:39.428676+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:39:39.428676+0000","last_undegraded":"2024-08-18T13:39:39.428676+0000","last_fullsized":"2024-08-18T13:39:39.428676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.430090+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:39:40.430090+0000","last_peered":"2024-08-18T13:39:40.430090+0000","last_clean":"2024-08-18T13:39:40.430090+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:39:40.430090+0000","last_undegraded":"2024-08-18T13:39:40.430090+0000","last_fullsized":"2024-08-18T13:39:40.430090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.443080+0000","last_change":"2024-08-18T13:39:39.439633+0000","last_active":"2024-08-18T13:39:39.443080+0000","last_peered":"2024-08-18T13:39:39.443080+0000","last_clean":"2024-08-18T13:39:39.443080+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:39:39.443080+0000","last_undegraded":"2024-08-18T13:39:39.443080+0000","last_fullsized":"2024-08-18T13:39:39.443080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423607+0000","last_change":"2024-08-18T13:39:35.407947+0000","last_active":"2024-08-18T13:39:39.423607+0000","last_peered":"2024-08-18T13:39:39.423607+0000","last_clean":"2024-08-18T13:39:39.423607+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:39:39.423607+0000","last_undegraded":"2024-08-18T13:39:39.423607+0000","last_fullsized":"2024-08-18T13:39:39.423607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429156+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:39:39.429156+0000","last_peered":"2024-08-18T13:39:39.429156+0000","last_clean":"2024-08-18T13:39:39.429156+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:39:39.429156+0000","last_undegraded":"2024-08-18T13:39:39.429156+0000","last_fullsized":"2024-08-18T13:39:39.429156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442892+0000","last_change":"2024-08-18T13:39:39.439913+0000","last_active":"2024-08-18T13:39:39.442892+0000","last_peered":"2024-08-18T13:39:39.442892+0000","last_clean":"2024-08-18T13:39:39.442892+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:39:39.442892+0000","last_undegraded":"2024-08-18T13:39:39.442892+0000","last_fullsized":"2024-08-18T13:39:39.442892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430332+0000","last_change":"2024-08-18T13:39:35.403254+0000","last_active":"2024-08-18T13:39:39.430332+0000","last_peered":"2024-08-18T13:39:39.430332+0000","last_clean":"2024-08-18T13:39:39.430332+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:39:39.430332+0000","last_undegraded":"2024-08-18T13:39:39.430332+0000","last_fullsized":"2024-08-18T13:39:39.430332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422689+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:39:39.422689+0000","last_peered":"2024-08-18T13:39:39.422689+0000","last_clean":"2024-08-18T13:39:39.422689+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:39:39.422689+0000","last_undegraded":"2024-08-18T13:39:39.422689+0000","last_fullsized":"2024-08-18T13:39:39.422689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457822+0000","last_change":"2024-08-18T13:39:39.457250+0000","last_active":"2024-08-18T13:39:39.457822+0000","last_peered":"2024-08-18T13:39:39.457822+0000","last_clean":"2024-08-18T13:39:39.457822+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:39:39.457822+0000","last_undegraded":"2024-08-18T13:39:39.457822+0000","last_fullsized":"2024-08-18T13:39:39.457822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425047+0000","last_change":"2024-08-18T13:39:35.401612+0000","last_active":"2024-08-18T13:39:39.425047+0000","last_peered":"2024-08-18T13:39:39.425047+0000","last_clean":"2024-08-18T13:39:39.425047+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:39:39.425047+0000","last_undegraded":"2024-08-18T13:39:39.425047+0000","last_fullsized":"2024-08-18T13:39:39.425047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450234+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:39:39.450234+0000","last_peered":"2024-08-18T13:39:39.450234+0000","last_clean":"2024-08-18T13:39:39.450234+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:39:39.450234+0000","last_undegraded":"2024-08-18T13:39:39.450234+0000","last_fullsized":"2024-08-18T13:39:39.450234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447688+0000","last_change":"2024-08-18T13:39:39.443503+0000","last_active":"2024-08-18T13:39:39.447688+0000","last_peered":"2024-08-18T13:39:39.447688+0000","last_clean":"2024-08-18T13:39:39.447688+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:39:39.447688+0000","last_undegraded":"2024-08-18T13:39:39.447688+0000","last_fullsized":"2024-08-18T13:39:39.447688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432976+0000","last_change":"2024-08-18T13:39:35.407907+0000","last_active":"2024-08-18T13:39:39.432976+0000","last_peered":"2024-08-18T13:39:39.432976+0000","last_clean":"2024-08-18T13:39:39.432976+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:39:39.432976+0000","last_undegraded":"2024-08-18T13:39:39.432976+0000","last_fullsized":"2024-08-18T13:39:39.432976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429181+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:39:40.429181+0000","last_peered":"2024-08-18T13:39:40.429181+0000","last_clean":"2024-08-18T13:39:40.429181+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:39:40.429181+0000","last_undegraded":"2024-08-18T13:39:40.429181+0000","last_fullsized":"2024-08-18T13:39:40.429181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454767+0000","last_change":"2024-08-18T13:39:39.453138+0000","last_active":"2024-08-18T13:39:39.454767+0000","last_peered":"2024-08-18T13:39:39.454767+0000","last_clean":"2024-08-18T13:39:39.454767+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:39:39.454767+0000","last_undegraded":"2024-08-18T13:39:39.454767+0000","last_fullsized":"2024-08-18T13:39:39.454767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424552+0000","last_change":"2024-08-18T13:39:35.404453+0000","last_active":"2024-08-18T13:39:39.424552+0000","last_peered":"2024-08-18T13:39:39.424552+0000","last_clean":"2024-08-18T13:39:39.424552+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:39:39.424552+0000","last_undegraded":"2024-08-18T13:39:39.424552+0000","last_fullsized":"2024-08-18T13:39:39.424552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:39:41.051281+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:39:41.051281+0000","last_peered":"2024-08-18T13:39:41.051281+0000","last_clean":"2024-08-18T13:39:41.051281+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:39:41.051281+0000","last_undegraded":"2024-08-18T13:39:41.051281+0000","last_fullsized":"2024-08-18T13:39:41.051281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450645+0000","last_change":"2024-08-18T13:39:39.449851+0000","last_active":"2024-08-18T13:39:39.450645+0000","last_peered":"2024-08-18T13:39:39.450645+0000","last_clean":"2024-08-18T13:39:39.450645+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:39:39.450645+0000","last_undegraded":"2024-08-18T13:39:39.450645+0000","last_fullsized":"2024-08-18T13:39:39.450645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424459+0000","last_change":"2024-08-18T13:39:35.400604+0000","last_active":"2024-08-18T13:39:39.424459+0000","last_peered":"2024-08-18T13:39:39.424459+0000","last_clean":"2024-08-18T13:39:39.424459+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:39:39.424459+0000","last_undegraded":"2024-08-18T13:39:39.424459+0000","last_fullsized":"2024-08-18T13:39:39.424459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450871+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:39:39.450871+0000","last_peered":"2024-08-18T13:39:39.450871+0000","last_clean":"2024-08-18T13:39:39.450871+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:39:39.450871+0000","last_undegraded":"2024-08-18T13:39:39.450871+0000","last_fullsized":"2024-08-18T13:39:39.450871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454096+0000","last_change":"2024-08-18T13:39:39.453599+0000","last_active":"2024-08-18T13:39:39.454096+0000","last_peered":"2024-08-18T13:39:39.454096+0000","last_clean":"2024-08-18T13:39:39.454096+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:39:39.454096+0000","last_undegraded":"2024-08-18T13:39:39.454096+0000","last_fullsized":"2024-08-18T13:39:39.454096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423160+0000","last_change":"2024-08-18T13:39:35.410167+0000","last_active":"2024-08-18T13:39:39.423160+0000","last_peered":"2024-08-18T13:39:39.423160+0000","last_clean":"2024-08-18T13:39:39.423160+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:39:39.423160+0000","last_undegraded":"2024-08-18T13:39:39.423160+0000","last_fullsized":"2024-08-18T13:39:39.423160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424551+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:39:39.424551+0000","last_peered":"2024-08-18T13:39:39.424551+0000","last_clean":"2024-08-18T13:39:39.424551+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:39:39.424551+0000","last_undegraded":"2024-08-18T13:39:39.424551+0000","last_fullsized":"2024-08-18T13:39:39.424551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444517+0000","last_change":"2024-08-18T13:39:39.439948+0000","last_active":"2024-08-18T13:39:39.444517+0000","last_peered":"2024-08-18T13:39:39.444517+0000","last_clean":"2024-08-18T13:39:39.444517+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:39:39.444517+0000","last_undegraded":"2024-08-18T13:39:39.444517+0000","last_fullsized":"2024-08-18T13:39:39.444517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422894+0000","last_change":"2024-08-18T13:39:35.407829+0000","last_active":"2024-08-18T13:39:39.422894+0000","last_peered":"2024-08-18T13:39:39.422894+0000","last_clean":"2024-08-18T13:39:39.422894+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:39:39.422894+0000","last_undegraded":"2024-08-18T13:39:39.422894+0000","last_fullsized":"2024-08-18T13:39:39.422894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417877+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:39:39.417877+0000","last_peered":"2024-08-18T13:39:39.417877+0000","last_clean":"2024-08-18T13:39:39.417877+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:39:39.417877+0000","last_undegraded":"2024-08-18T13:39:39.417877+0000","last_fullsized":"2024-08-18T13:39:39.417877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445479+0000","last_change":"2024-08-18T13:39:39.442372+0000","last_active":"2024-08-18T13:39:39.445479+0000","last_peered":"2024-08-18T13:39:39.445479+0000","last_clean":"2024-08-18T13:39:39.445479+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:39:39.445479+0000","last_undegraded":"2024-08-18T13:39:39.445479+0000","last_fullsized":"2024-08-18T13:39:39.445479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421862+0000","last_change":"2024-08-18T13:39:35.400384+0000","last_active":"2024-08-18T13:39:39.421862+0000","last_peered":"2024-08-18T13:39:39.421862+0000","last_clean":"2024-08-18T13:39:39.421862+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:39:39.421862+0000","last_undegraded":"2024-08-18T13:39:39.421862+0000","last_fullsized":"2024-08-18T13:39:39.421862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415832+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:39:39.415832+0000","last_peered":"2024-08-18T13:39:39.415832+0000","last_clean":"2024-08-18T13:39:39.415832+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:39:39.415832+0000","last_undegraded":"2024-08-18T13:39:39.415832+0000","last_fullsized":"2024-08-18T13:39:39.415832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446859+0000","last_change":"2024-08-18T13:39:39.446401+0000","last_active":"2024-08-18T13:39:39.446859+0000","last_peered":"2024-08-18T13:39:39.446859+0000","last_clean":"2024-08-18T13:39:39.446859+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:39:39.446859+0000","last_undegraded":"2024-08-18T13:39:39.446859+0000","last_fullsized":"2024-08-18T13:39:39.446859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424307+0000","last_change":"2024-08-18T13:39:35.410127+0000","last_active":"2024-08-18T13:39:39.424307+0000","last_peered":"2024-08-18T13:39:39.424307+0000","last_clean":"2024-08-18T13:39:39.424307+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:39:39.424307+0000","last_undegraded":"2024-08-18T13:39:39.424307+0000","last_fullsized":"2024-08-18T13:39:39.424307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450338+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:39:39.450338+0000","last_peered":"2024-08-18T13:39:39.450338+0000","last_clean":"2024-08-18T13:39:39.450338+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:39:39.450338+0000","last_undegraded":"2024-08-18T13:39:39.450338+0000","last_fullsized":"2024-08-18T13:39:39.450338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454861+0000","last_change":"2024-08-18T13:39:39.453901+0000","last_active":"2024-08-18T13:39:39.454861+0000","last_peered":"2024-08-18T13:39:39.454861+0000","last_clean":"2024-08-18T13:39:39.454861+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:39:39.454861+0000","last_undegraded":"2024-08-18T13:39:39.454861+0000","last_fullsized":"2024-08-18T13:39:39.454861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430338+0000","last_change":"2024-08-18T13:39:35.412156+0000","last_active":"2024-08-18T13:39:39.430338+0000","last_peered":"2024-08-18T13:39:39.430338+0000","last_clean":"2024-08-18T13:39:39.430338+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:39:39.430338+0000","last_undegraded":"2024-08-18T13:39:39.430338+0000","last_fullsized":"2024-08-18T13:39:39.430338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417339+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:39:39.417339+0000","last_peered":"2024-08-18T13:39:39.417339+0000","last_clean":"2024-08-18T13:39:39.417339+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:39:39.417339+0000","last_undegraded":"2024-08-18T13:39:39.417339+0000","last_fullsized":"2024-08-18T13:39:39.417339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450727+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:39:39.450727+0000","last_peered":"2024-08-18T13:39:39.450727+0000","last_clean":"2024-08-18T13:39:39.450727+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:39:39.450727+0000","last_undegraded":"2024-08-18T13:39:39.450727+0000","last_fullsized":"2024-08-18T13:39:39.450727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454332+0000","last_change":"2024-08-18T13:39:39.453360+0000","last_active":"2024-08-18T13:39:39.454332+0000","last_peered":"2024-08-18T13:39:39.454332+0000","last_clean":"2024-08-18T13:39:39.454332+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:39:39.454332+0000","last_undegraded":"2024-08-18T13:39:39.454332+0000","last_fullsized":"2024-08-18T13:39:39.454332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450556+0000","last_change":"2024-08-18T13:39:35.421765+0000","last_active":"2024-08-18T13:39:39.450556+0000","last_peered":"2024-08-18T13:39:39.450556+0000","last_clean":"2024-08-18T13:39:39.450556+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:39:39.450556+0000","last_undegraded":"2024-08-18T13:39:39.450556+0000","last_fullsized":"2024-08-18T13:39:39.450556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416404+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:39:39.416404+0000","last_peered":"2024-08-18T13:39:39.416404+0000","last_clean":"2024-08-18T13:39:39.416404+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:39:39.416404+0000","last_undegraded":"2024-08-18T13:39:39.416404+0000","last_fullsized":"2024-08-18T13:39:39.416404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450222+0000","last_change":"2024-08-18T13:39:39.449653+0000","last_active":"2024-08-18T13:39:39.450222+0000","last_peered":"2024-08-18T13:39:39.450222+0000","last_clean":"2024-08-18T13:39:39.450222+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:39:39.450222+0000","last_undegraded":"2024-08-18T13:39:39.450222+0000","last_fullsized":"2024-08-18T13:39:39.450222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422710+0000","last_change":"2024-08-18T13:39:35.407439+0000","last_active":"2024-08-18T13:39:39.422710+0000","last_peered":"2024-08-18T13:39:39.422710+0000","last_clean":"2024-08-18T13:39:39.422710+0000","last_became_active":"2024-08-18T13:39:35.407128+0000","last_became_peered":"2024-08-18T13:39:35.407128+0000","last_unstale":"2024-08-18T13:39:39.422710+0000","last_undegraded":"2024-08-18T13:39:39.422710+0000","last_fullsized":"2024-08-18T13:39:39.422710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431851+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:39:40.431851+0000","last_peered":"2024-08-18T13:39:40.431851+0000","last_clean":"2024-08-18T13:39:40.431851+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:39:40.431851+0000","last_undegraded":"2024-08-18T13:39:40.431851+0000","last_fullsized":"2024-08-18T13:39:40.431851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.445844+0000","last_change":"2024-08-18T13:39:39.443848+0000","last_active":"2024-08-18T13:39:39.445844+0000","last_peered":"2024-08-18T13:39:39.445844+0000","last_clean":"2024-08-18T13:39:39.445844+0000","last_became_active":"2024-08-18T13:39:39.443456+0000","last_became_peered":"2024-08-18T13:39:39.443456+0000","last_unstale":"2024-08-18T13:39:39.445844+0000","last_undegraded":"2024-08-18T13:39:39.445844+0000","last_fullsized":"2024-08-18T13:39:39.445844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:27.994932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450757+0000","last_change":"2024-08-18T13:39:35.421612+0000","last_active":"2024-08-18T13:39:39.450757+0000","last_peered":"2024-08-18T13:39:39.450757+0000","last_clean":"2024-08-18T13:39:39.450757+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:39:39.450757+0000","last_undegraded":"2024-08-18T13:39:39.450757+0000","last_fullsized":"2024-08-18T13:39:39.450757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433442+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:39:40.433442+0000","last_peered":"2024-08-18T13:39:40.433442+0000","last_clean":"2024-08-18T13:39:40.433442+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:39:40.433442+0000","last_undegraded":"2024-08-18T13:39:40.433442+0000","last_fullsized":"2024-08-18T13:39:40.433442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454815+0000","last_change":"2024-08-18T13:39:39.453520+0000","last_active":"2024-08-18T13:39:39.454815+0000","last_peered":"2024-08-18T13:39:39.454815+0000","last_clean":"2024-08-18T13:39:39.454815+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:39:39.454815+0000","last_undegraded":"2024-08-18T13:39:39.454815+0000","last_fullsized":"2024-08-18T13:39:39.454815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450976+0000","last_change":"2024-08-18T13:39:35.421465+0000","last_active":"2024-08-18T13:39:39.450976+0000","last_peered":"2024-08-18T13:39:39.450976+0000","last_clean":"2024-08-18T13:39:39.450976+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:39:39.450976+0000","last_undegraded":"2024-08-18T13:39:39.450976+0000","last_fullsized":"2024-08-18T13:39:39.450976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429394+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:39:40.429394+0000","last_peered":"2024-08-18T13:39:40.429394+0000","last_clean":"2024-08-18T13:39:40.429394+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:39:40.429394+0000","last_undegraded":"2024-08-18T13:39:40.429394+0000","last_fullsized":"2024-08-18T13:39:40.429394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.448067+0000","last_change":"2024-08-18T13:39:39.446029+0000","last_active":"2024-08-18T13:39:39.448067+0000","last_peered":"2024-08-18T13:39:39.448067+0000","last_clean":"2024-08-18T13:39:39.448067+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:39:39.448067+0000","last_undegraded":"2024-08-18T13:39:39.448067+0000","last_fullsized":"2024-08-18T13:39:39.448067+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450468+0000","last_change":"2024-08-18T13:39:35.417094+0000","last_active":"2024-08-18T13:39:39.450468+0000","last_peered":"2024-08-18T13:39:39.450468+0000","last_clean":"2024-08-18T13:39:39.450468+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:39:39.450468+0000","last_undegraded":"2024-08-18T13:39:39.450468+0000","last_fullsized":"2024-08-18T13:39:39.450468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.436721+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:39:40.436721+0000","last_peered":"2024-08-18T13:39:40.436721+0000","last_clean":"2024-08-18T13:39:40.436721+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:39:40.436721+0000","last_undegraded":"2024-08-18T13:39:40.436721+0000","last_fullsized":"2024-08-18T13:39:40.436721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.437633+0000","last_change":"2024-08-18T13:39:39.436992+0000","last_active":"2024-08-18T13:39:39.437633+0000","last_peered":"2024-08-18T13:39:39.437633+0000","last_clean":"2024-08-18T13:39:39.437633+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:39:39.437633+0000","last_undegraded":"2024-08-18T13:39:39.437633+0000","last_fullsized":"2024-08-18T13:39:39.437633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427745+0000","last_change":"2024-08-18T13:39:35.411315+0000","last_active":"2024-08-18T13:39:39.427745+0000","last_peered":"2024-08-18T13:39:39.427745+0000","last_clean":"2024-08-18T13:39:39.427745+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:39:39.427745+0000","last_undegraded":"2024-08-18T13:39:39.427745+0000","last_fullsized":"2024-08-18T13:39:39.427745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.428272+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:39:39.428272+0000","last_peered":"2024-08-18T13:39:39.428272+0000","last_clean":"2024-08-18T13:39:39.428272+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:39:39.428272+0000","last_undegraded":"2024-08-18T13:39:39.428272+0000","last_fullsized":"2024-08-18T13:39:39.428272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.458299+0000","last_change":"2024-08-18T13:39:39.456955+0000","last_active":"2024-08-18T13:39:39.458299+0000","last_peered":"2024-08-18T13:39:39.458299+0000","last_clean":"2024-08-18T13:39:39.458299+0000","last_became_active":"2024-08-18T13:39:39.456534+0000","last_became_peered":"2024-08-18T13:39:39.456534+0000","last_unstale":"2024-08-18T13:39:39.458299+0000","last_undegraded":"2024-08-18T13:39:39.458299+0000","last_fullsized":"2024-08-18T13:39:39.458299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:26:42.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450840+0000","last_change":"2024-08-18T13:39:35.417348+0000","last_active":"2024-08-18T13:39:39.450840+0000","last_peered":"2024-08-18T13:39:39.450840+0000","last_clean":"2024-08-18T13:39:39.450840+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:39:39.450840+0000","last_undegraded":"2024-08-18T13:39:39.450840+0000","last_fullsized":"2024-08-18T13:39:39.450840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417725+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:39:39.417725+0000","last_peered":"2024-08-18T13:39:39.417725+0000","last_clean":"2024-08-18T13:39:39.417725+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:39:39.417725+0000","last_undegraded":"2024-08-18T13:39:39.417725+0000","last_fullsized":"2024-08-18T13:39:39.417725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454724+0000","last_change":"2024-08-18T13:39:39.453682+0000","last_active":"2024-08-18T13:39:39.454724+0000","last_peered":"2024-08-18T13:39:39.454724+0000","last_clean":"2024-08-18T13:39:39.454724+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:39:39.454724+0000","last_undegraded":"2024-08-18T13:39:39.454724+0000","last_fullsized":"2024-08-18T13:39:39.454724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427706+0000","last_change":"2024-08-18T13:39:35.407127+0000","last_active":"2024-08-18T13:39:39.427706+0000","last_peered":"2024-08-18T13:39:39.427706+0000","last_clean":"2024-08-18T13:39:39.427706+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:39:39.427706+0000","last_undegraded":"2024-08-18T13:39:39.427706+0000","last_fullsized":"2024-08-18T13:39:39.427706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.431523+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:39:40.431523+0000","last_peered":"2024-08-18T13:39:40.431523+0000","last_clean":"2024-08-18T13:39:40.431523+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:39:40.431523+0000","last_undegraded":"2024-08-18T13:39:40.431523+0000","last_fullsized":"2024-08-18T13:39:40.431523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.457313+0000","last_change":"2024-08-18T13:39:39.456626+0000","last_active":"2024-08-18T13:39:39.457313+0000","last_peered":"2024-08-18T13:39:39.457313+0000","last_clean":"2024-08-18T13:39:39.457313+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:39:39.457313+0000","last_undegraded":"2024-08-18T13:39:39.457313+0000","last_fullsized":"2024-08-18T13:39:39.457313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450395+0000","last_change":"2024-08-18T13:39:35.417693+0000","last_active":"2024-08-18T13:39:39.450395+0000","last_peered":"2024-08-18T13:39:39.450395+0000","last_clean":"2024-08-18T13:39:39.450395+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:39:39.450395+0000","last_undegraded":"2024-08-18T13:39:39.450395+0000","last_fullsized":"2024-08-18T13:39:39.450395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-08-18T13:40:09.449886+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:40:09.449886+0000","last_peered":"2024-08-18T13:40:09.449886+0000","last_clean":"2024-08-18T13:40:09.449886+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:40:09.449886+0000","last_undegraded":"2024-08-18T13:40:09.449886+0000","last_fullsized":"2024-08-18T13:40:09.449886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4207,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.440639+0000","last_change":"2024-08-18T13:39:39.438064+0000","last_active":"2024-08-18T13:39:39.440639+0000","last_peered":"2024-08-18T13:39:39.440639+0000","last_clean":"2024-08-18T13:39:39.440639+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:39:39.440639+0000","last_undegraded":"2024-08-18T13:39:39.440639+0000","last_fullsized":"2024-08-18T13:39:39.440639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421839+0000","last_change":"2024-08-18T13:39:35.406398+0000","last_active":"2024-08-18T13:39:39.421839+0000","last_peered":"2024-08-18T13:39:39.421839+0000","last_clean":"2024-08-18T13:39:39.421839+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:39:39.421839+0000","last_undegraded":"2024-08-18T13:39:39.421839+0000","last_fullsized":"2024-08-18T13:39:39.421839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.433257+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:39:40.433257+0000","last_peered":"2024-08-18T13:39:40.433257+0000","last_clean":"2024-08-18T13:39:40.433257+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:39:40.433257+0000","last_undegraded":"2024-08-18T13:39:40.433257+0000","last_fullsized":"2024-08-18T13:39:40.433257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.444385+0000","last_change":"2024-08-18T13:39:39.439962+0000","last_active":"2024-08-18T13:39:39.444385+0000","last_peered":"2024-08-18T13:39:39.444385+0000","last_clean":"2024-08-18T13:39:39.444385+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:39:39.444385+0000","last_undegraded":"2024-08-18T13:39:39.444385+0000","last_fullsized":"2024-08-18T13:39:39.444385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.432863+0000","last_change":"2024-08-18T13:39:35.411067+0000","last_active":"2024-08-18T13:39:39.432863+0000","last_peered":"2024-08-18T13:39:39.432863+0000","last_clean":"2024-08-18T13:39:39.432863+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:39:39.432863+0000","last_undegraded":"2024-08-18T13:39:39.432863+0000","last_fullsized":"2024-08-18T13:39:39.432863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422811+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:39:39.422811+0000","last_peered":"2024-08-18T13:39:39.422811+0000","last_clean":"2024-08-18T13:39:39.422811+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:39:39.422811+0000","last_undegraded":"2024-08-18T13:39:39.422811+0000","last_fullsized":"2024-08-18T13:39:39.422811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.447069+0000","last_change":"2024-08-18T13:39:39.443938+0000","last_active":"2024-08-18T13:39:39.447069+0000","last_peered":"2024-08-18T13:39:39.447069+0000","last_clean":"2024-08-18T13:39:39.447069+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:39:39.447069+0000","last_undegraded":"2024-08-18T13:39:39.447069+0000","last_fullsized":"2024-08-18T13:39:39.447069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422769+0000","last_change":"2024-08-18T13:39:35.409517+0000","last_active":"2024-08-18T13:39:39.422769+0000","last_peered":"2024-08-18T13:39:39.422769+0000","last_clean":"2024-08-18T13:39:39.422769+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:39:39.422769+0000","last_undegraded":"2024-08-18T13:39:39.422769+0000","last_fullsized":"2024-08-18T13:39:39.422769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429841+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:39:39.429841+0000","last_peered":"2024-08-18T13:39:39.429841+0000","last_clean":"2024-08-18T13:39:39.429841+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:39:39.429841+0000","last_undegraded":"2024-08-18T13:39:39.429841+0000","last_fullsized":"2024-08-18T13:39:39.429841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450637+0000","last_change":"2024-08-18T13:39:39.449671+0000","last_active":"2024-08-18T13:39:39.450637+0000","last_peered":"2024-08-18T13:39:39.450637+0000","last_clean":"2024-08-18T13:39:39.450637+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:39:39.450637+0000","last_undegraded":"2024-08-18T13:39:39.450637+0000","last_fullsized":"2024-08-18T13:39:39.450637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.454824+0000","last_change":"2024-08-18T13:39:39.453907+0000","last_active":"2024-08-18T13:39:39.454824+0000","last_peered":"2024-08-18T13:39:39.454824+0000","last_clean":"2024-08-18T13:39:39.454824+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:39:39.454824+0000","last_undegraded":"2024-08-18T13:39:39.454824+0000","last_fullsized":"2024-08-18T13:39:39.454824+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422264+0000","last_change":"2024-08-18T13:39:35.401997+0000","last_active":"2024-08-18T13:39:39.422264+0000","last_peered":"2024-08-18T13:39:39.422264+0000","last_clean":"2024-08-18T13:39:39.422264+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:39:39.422264+0000","last_undegraded":"2024-08-18T13:39:39.422264+0000","last_fullsized":"2024-08-18T13:39:39.422264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418448+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:39:39.418448+0000","last_peered":"2024-08-18T13:39:39.418448+0000","last_clean":"2024-08-18T13:39:39.418448+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:39:39.418448+0000","last_undegraded":"2024-08-18T13:39:39.418448+0000","last_fullsized":"2024-08-18T13:39:39.418448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.442977+0000","last_change":"2024-08-18T13:39:39.440811+0000","last_active":"2024-08-18T13:39:39.442977+0000","last_peered":"2024-08-18T13:39:39.442977+0000","last_clean":"2024-08-18T13:39:39.442977+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:39:39.442977+0000","last_undegraded":"2024-08-18T13:39:39.442977+0000","last_fullsized":"2024-08-18T13:39:39.442977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430210+0000","last_change":"2024-08-18T13:39:35.412127+0000","last_active":"2024-08-18T13:39:39.430210+0000","last_peered":"2024-08-18T13:39:39.430210+0000","last_clean":"2024-08-18T13:39:39.430210+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:39:39.430210+0000","last_undegraded":"2024-08-18T13:39:39.430210+0000","last_fullsized":"2024-08-18T13:39:39.430210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423847+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:39:39.423847+0000","last_peered":"2024-08-18T13:39:39.423847+0000","last_clean":"2024-08-18T13:39:39.423847+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:39:39.423847+0000","last_undegraded":"2024-08-18T13:39:39.423847+0000","last_fullsized":"2024-08-18T13:39:39.423847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.429678+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:39:40.429678+0000","last_peered":"2024-08-18T13:39:40.429678+0000","last_clean":"2024-08-18T13:39:40.429678+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:39:40.429678+0000","last_undegraded":"2024-08-18T13:39:40.429678+0000","last_fullsized":"2024-08-18T13:39:40.429678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429574+0000","last_change":"2024-08-18T13:39:35.407995+0000","last_active":"2024-08-18T13:39:39.429574+0000","last_peered":"2024-08-18T13:39:39.429574+0000","last_clean":"2024-08-18T13:39:39.429574+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:39:39.429574+0000","last_undegraded":"2024-08-18T13:39:39.429574+0000","last_fullsized":"2024-08-18T13:39:39.429574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.438294+0000","last_change":"2024-08-18T13:39:39.437542+0000","last_active":"2024-08-18T13:39:39.438294+0000","last_peered":"2024-08-18T13:39:39.438294+0000","last_clean":"2024-08-18T13:39:39.438294+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:39:39.438294+0000","last_undegraded":"2024-08-18T13:39:39.438294+0000","last_fullsized":"2024-08-18T13:39:39.438294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:40.434310+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:39:40.434310+0000","last_peered":"2024-08-18T13:39:40.434310+0000","last_clean":"2024-08-18T13:39:40.434310+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:39:40.434310+0000","last_undegraded":"2024-08-18T13:39:40.434310+0000","last_fullsized":"2024-08-18T13:39:40.434310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422760+0000","last_change":"2024-08-18T13:39:35.403759+0000","last_active":"2024-08-18T13:39:39.422760+0000","last_peered":"2024-08-18T13:39:39.422760+0000","last_clean":"2024-08-18T13:39:39.422760+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:39:39.422760+0000","last_undegraded":"2024-08-18T13:39:39.422760+0000","last_fullsized":"2024-08-18T13:39:39.422760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.446432+0000","last_change":"2024-08-18T13:39:39.441201+0000","last_active":"2024-08-18T13:39:39.446432+0000","last_peered":"2024-08-18T13:39:39.446432+0000","last_clean":"2024-08-18T13:39:39.446432+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:39:39.446432+0000","last_undegraded":"2024-08-18T13:39:39.446432+0000","last_fullsized":"2024-08-18T13:39:39.446432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427616+0000","last_change":"2024-08-18T13:39:35.413292+0000","last_active":"2024-08-18T13:39:39.427616+0000","last_peered":"2024-08-18T13:39:39.427616+0000","last_clean":"2024-08-18T13:39:39.427616+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:39:39.427616+0000","last_undegraded":"2024-08-18T13:39:39.427616+0000","last_fullsized":"2024-08-18T13:39:39.427616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.415812+0000","last_change":"2024-08-18T13:39:35.401196+0000","last_active":"2024-08-18T13:39:39.415812+0000","last_peered":"2024-08-18T13:39:39.415812+0000","last_clean":"2024-08-18T13:39:39.415812+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:39:39.415812+0000","last_undegraded":"2024-08-18T13:39:39.415812+0000","last_fullsized":"2024-08-18T13:39:39.415812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417577+0000","last_change":"2024-08-18T13:39:35.406603+0000","last_active":"2024-08-18T13:39:39.417577+0000","last_peered":"2024-08-18T13:39:39.417577+0000","last_clean":"2024-08-18T13:39:39.417577+0000","last_became_active":"2024-08-18T13:39:35.406331+0000","last_became_peered":"2024-08-18T13:39:35.406331+0000","last_unstale":"2024-08-18T13:39:39.417577+0000","last_undegraded":"2024-08-18T13:39:39.417577+0000","last_fullsized":"2024-08-18T13:39:39.417577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:21:12.232172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429494+0000","last_change":"2024-08-18T13:39:35.404051+0000","last_active":"2024-08-18T13:39:39.429494+0000","last_peered":"2024-08-18T13:39:39.429494+0000","last_clean":"2024-08-18T13:39:39.429494+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:39:39.429494+0000","last_undegraded":"2024-08-18T13:39:39.429494+0000","last_fullsized":"2024-08-18T13:39:39.429494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422959+0000","last_change":"2024-08-18T13:39:35.409288+0000","last_active":"2024-08-18T13:39:39.422959+0000","last_peered":"2024-08-18T13:39:39.422959+0000","last_clean":"2024-08-18T13:39:39.422959+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:39:39.422959+0000","last_undegraded":"2024-08-18T13:39:39.422959+0000","last_fullsized":"2024-08-18T13:39:39.422959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424121+0000","last_change":"2024-08-18T13:39:35.403051+0000","last_active":"2024-08-18T13:39:39.424121+0000","last_peered":"2024-08-18T13:39:39.424121+0000","last_clean":"2024-08-18T13:39:39.424121+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:39:39.424121+0000","last_undegraded":"2024-08-18T13:39:39.424121+0000","last_fullsized":"2024-08-18T13:39:39.424121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.430049+0000","last_change":"2024-08-18T13:39:35.412281+0000","last_active":"2024-08-18T13:39:39.430049+0000","last_peered":"2024-08-18T13:39:39.430049+0000","last_clean":"2024-08-18T13:39:39.430049+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:39:39.430049+0000","last_undegraded":"2024-08-18T13:39:39.430049+0000","last_fullsized":"2024-08-18T13:39:39.430049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.424466+0000","last_change":"2024-08-18T13:39:35.409220+0000","last_active":"2024-08-18T13:39:39.424466+0000","last_peered":"2024-08-18T13:39:39.424466+0000","last_clean":"2024-08-18T13:39:39.424466+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:39:39.424466+0000","last_undegraded":"2024-08-18T13:39:39.424466+0000","last_fullsized":"2024-08-18T13:39:39.424466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418253+0000","last_change":"2024-08-18T13:39:35.401759+0000","last_active":"2024-08-18T13:39:39.418253+0000","last_peered":"2024-08-18T13:39:39.418253+0000","last_clean":"2024-08-18T13:39:39.418253+0000","last_became_active":"2024-08-18T13:39:35.401476+0000","last_became_peered":"2024-08-18T13:39:35.401476+0000","last_unstale":"2024-08-18T13:39:39.418253+0000","last_undegraded":"2024-08-18T13:39:39.418253+0000","last_fullsized":"2024-08-18T13:39:39.418253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:04:43.997597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423669+0000","last_change":"2024-08-18T13:39:35.408967+0000","last_active":"2024-08-18T13:39:39.423669+0000","last_peered":"2024-08-18T13:39:39.423669+0000","last_clean":"2024-08-18T13:39:39.423669+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:39:39.423669+0000","last_undegraded":"2024-08-18T13:39:39.423669+0000","last_fullsized":"2024-08-18T13:39:39.423669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417410+0000","last_change":"2024-08-18T13:39:35.407218+0000","last_active":"2024-08-18T13:39:39.417410+0000","last_peered":"2024-08-18T13:39:39.417410+0000","last_clean":"2024-08-18T13:39:39.417410+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:39:39.417410+0000","last_undegraded":"2024-08-18T13:39:39.417410+0000","last_fullsized":"2024-08-18T13:39:39.417410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429387+0000","last_change":"2024-08-18T13:39:35.407339+0000","last_active":"2024-08-18T13:39:39.429387+0000","last_peered":"2024-08-18T13:39:39.429387+0000","last_clean":"2024-08-18T13:39:39.429387+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:39:39.429387+0000","last_undegraded":"2024-08-18T13:39:39.429387+0000","last_fullsized":"2024-08-18T13:39:39.429387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429062+0000","last_change":"2024-08-18T13:39:35.408647+0000","last_active":"2024-08-18T13:39:39.429062+0000","last_peered":"2024-08-18T13:39:39.429062+0000","last_clean":"2024-08-18T13:39:39.429062+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:39:39.429062+0000","last_undegraded":"2024-08-18T13:39:39.429062+0000","last_fullsized":"2024-08-18T13:39:39.429062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450837+0000","last_change":"2024-08-18T13:39:35.418000+0000","last_active":"2024-08-18T13:39:39.450837+0000","last_peered":"2024-08-18T13:39:39.450837+0000","last_clean":"2024-08-18T13:39:39.450837+0000","last_became_active":"2024-08-18T13:39:35.417661+0000","last_became_peered":"2024-08-18T13:39:35.417661+0000","last_unstale":"2024-08-18T13:39:39.450837+0000","last_undegraded":"2024-08-18T13:39:39.450837+0000","last_fullsized":"2024-08-18T13:39:39.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:53:44.739585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.416124+0000","last_change":"2024-08-18T13:39:35.405425+0000","last_active":"2024-08-18T13:39:39.416124+0000","last_peered":"2024-08-18T13:39:39.416124+0000","last_clean":"2024-08-18T13:39:39.416124+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:39:39.416124+0000","last_undegraded":"2024-08-18T13:39:39.416124+0000","last_fullsized":"2024-08-18T13:39:39.416124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423886+0000","last_change":"2024-08-18T13:39:35.400889+0000","last_active":"2024-08-18T13:39:39.423886+0000","last_peered":"2024-08-18T13:39:39.423886+0000","last_clean":"2024-08-18T13:39:39.423886+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:39:39.423886+0000","last_undegraded":"2024-08-18T13:39:39.423886+0000","last_fullsized":"2024-08-18T13:39:39.423886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.418057+0000","last_change":"2024-08-18T13:39:35.407634+0000","last_active":"2024-08-18T13:39:39.418057+0000","last_peered":"2024-08-18T13:39:39.418057+0000","last_clean":"2024-08-18T13:39:39.418057+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:39:39.418057+0000","last_undegraded":"2024-08-18T13:39:39.418057+0000","last_fullsized":"2024-08-18T13:39:39.418057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422473+0000","last_change":"2024-08-18T13:39:35.397509+0000","last_active":"2024-08-18T13:39:39.422473+0000","last_peered":"2024-08-18T13:39:39.422473+0000","last_clean":"2024-08-18T13:39:39.422473+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:39:39.422473+0000","last_undegraded":"2024-08-18T13:39:39.422473+0000","last_fullsized":"2024-08-18T13:39:39.422473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425330+0000","last_change":"2024-08-18T13:39:35.404533+0000","last_active":"2024-08-18T13:39:39.425330+0000","last_peered":"2024-08-18T13:39:39.425330+0000","last_clean":"2024-08-18T13:39:39.425330+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:39:39.425330+0000","last_undegraded":"2024-08-18T13:39:39.425330+0000","last_fullsized":"2024-08-18T13:39:39.425330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425567+0000","last_change":"2024-08-18T13:39:35.404839+0000","last_active":"2024-08-18T13:39:39.425567+0000","last_peered":"2024-08-18T13:39:39.425567+0000","last_clean":"2024-08-18T13:39:39.425567+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:39:39.425567+0000","last_undegraded":"2024-08-18T13:39:39.425567+0000","last_fullsized":"2024-08-18T13:39:39.425567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.421633+0000","last_change":"2024-08-18T13:39:35.405867+0000","last_active":"2024-08-18T13:39:39.421633+0000","last_peered":"2024-08-18T13:39:39.421633+0000","last_clean":"2024-08-18T13:39:39.421633+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:39:39.421633+0000","last_undegraded":"2024-08-18T13:39:39.421633+0000","last_fullsized":"2024-08-18T13:39:39.421633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429697+0000","last_change":"2024-08-18T13:39:35.411173+0000","last_active":"2024-08-18T13:39:39.429697+0000","last_peered":"2024-08-18T13:39:39.429697+0000","last_clean":"2024-08-18T13:39:39.429697+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:39:39.429697+0000","last_undegraded":"2024-08-18T13:39:39.429697+0000","last_fullsized":"2024-08-18T13:39:39.429697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.429962+0000","last_change":"2024-08-18T13:39:35.418013+0000","last_active":"2024-08-18T13:39:39.429962+0000","last_peered":"2024-08-18T13:39:39.429962+0000","last_clean":"2024-08-18T13:39:39.429962+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:39:39.429962+0000","last_undegraded":"2024-08-18T13:39:39.429962+0000","last_fullsized":"2024-08-18T13:39:39.429962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450803+0000","last_change":"2024-08-18T13:39:35.421633+0000","last_active":"2024-08-18T13:39:39.450803+0000","last_peered":"2024-08-18T13:39:39.450803+0000","last_clean":"2024-08-18T13:39:39.450803+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:39:39.450803+0000","last_undegraded":"2024-08-18T13:39:39.450803+0000","last_fullsized":"2024-08-18T13:39:39.450803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.425234+0000","last_change":"2024-08-18T13:39:35.411443+0000","last_active":"2024-08-18T13:39:39.425234+0000","last_peered":"2024-08-18T13:39:39.425234+0000","last_clean":"2024-08-18T13:39:39.425234+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:39:39.425234+0000","last_undegraded":"2024-08-18T13:39:39.425234+0000","last_fullsized":"2024-08-18T13:39:39.425234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450366+0000","last_change":"2024-08-18T13:39:35.415548+0000","last_active":"2024-08-18T13:39:39.450366+0000","last_peered":"2024-08-18T13:39:39.450366+0000","last_clean":"2024-08-18T13:39:39.450366+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:39:39.450366+0000","last_undegraded":"2024-08-18T13:39:39.450366+0000","last_fullsized":"2024-08-18T13:39:39.450366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.417246+0000","last_change":"2024-08-18T13:39:35.410795+0000","last_active":"2024-08-18T13:39:39.417246+0000","last_peered":"2024-08-18T13:39:39.417246+0000","last_clean":"2024-08-18T13:39:39.417246+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:39:39.417246+0000","last_undegraded":"2024-08-18T13:39:39.417246+0000","last_fullsized":"2024-08-18T13:39:39.417246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450710+0000","last_change":"2024-08-18T13:39:35.417550+0000","last_active":"2024-08-18T13:39:39.450710+0000","last_peered":"2024-08-18T13:39:39.450710+0000","last_clean":"2024-08-18T13:39:39.450710+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:39:39.450710+0000","last_undegraded":"2024-08-18T13:39:39.450710+0000","last_fullsized":"2024-08-18T13:39:39.450710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.422555+0000","last_change":"2024-08-18T13:39:35.410002+0000","last_active":"2024-08-18T13:39:39.422555+0000","last_peered":"2024-08-18T13:39:39.422555+0000","last_clean":"2024-08-18T13:39:39.422555+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:39:39.422555+0000","last_undegraded":"2024-08-18T13:39:39.422555+0000","last_fullsized":"2024-08-18T13:39:39.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.427751+0000","last_change":"2024-08-18T13:39:35.406627+0000","last_active":"2024-08-18T13:39:39.427751+0000","last_peered":"2024-08-18T13:39:39.427751+0000","last_clean":"2024-08-18T13:39:39.427751+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:39:39.427751+0000","last_undegraded":"2024-08-18T13:39:39.427751+0000","last_fullsized":"2024-08-18T13:39:39.427751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.423078+0000","last_change":"2024-08-18T13:39:35.405750+0000","last_active":"2024-08-18T13:39:39.423078+0000","last_peered":"2024-08-18T13:39:39.423078+0000","last_clean":"2024-08-18T13:39:39.423078+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:39:39.423078+0000","last_undegraded":"2024-08-18T13:39:39.423078+0000","last_fullsized":"2024-08-18T13:39:39.423078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-08-18T13:39:39.450591+0000","last_change":"2024-08-18T13:39:35.417195+0000","last_active":"2024-08-18T13:39:39.450591+0000","last_peered":"2024-08-18T13:39:39.450591+0000","last_clean":"2024-08-18T13:39:39.450591+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:39:39.450591+0000","last_undegraded":"2024-08-18T13:39:39.450591+0000","last_fullsized":"2024-08-18T13:39:39.450591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4993,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302352,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069488,"statfs":{"total":96636764160,"available":96327155712,"internally_reserved":0,"allocated":856064,"data_stored":693001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":270336,"data_stored":102189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":270336,"data_stored":106270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":258048,"data_stored":102063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]}]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301776,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070064,"statfs":{"total":96636764160,"available":96327745536,"internally_reserved":0,"allocated":266240,"data_stored":102223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":864256,"data_stored":693057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301772,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070068,"statfs":{"total":96636764160,"available":96327749632,"internally_reserved":0,"allocated":262144,"data_stored":102153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:40:15.444 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:40:15.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1f54a9700 1 Processor -- start 2024-08-18T13:40:15.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1f54a9700 1 -- start start 2024-08-18T13:40:15.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1f54a9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff1f00c8200 0x7ff1f00673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1f54a9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f006ee60 con 0x7ff1f0153690 2024-08-18T13:40:15.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1f54a9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f006f730 con 0x7ff1f006f2f0 2024-08-18T13:40:15.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1f54a9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f015a640 con 0x7ff1f00c8200 2024-08-18T13:40:15.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1effff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff1f0153690 0x7ff1f00c22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53936/0 (socket says 172.21.15.118:53936) 2024-08-18T13:40:15.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1effff700 1 -- 172.21.15.118:0/2217684714 learned_addr learned my addr 172.21.15.118:0/2217684714 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.550+0000 7ff1ef7fe700 1 --2- 172.21.15.118:0/2217684714 >> v2:172.21.15.178:3300/0 conn(0x7ff1f00c8200 0x7ff1f00673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1eeffd700 1 -- 172.21.15.118:0/2217684714 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311135978 0 0) 0x7ff1f006ee60 con 0x7ff1f0153690 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1eeffd700 1 -- 172.21.15.118:0/2217684714 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d8003570 con 0x7ff1f0153690 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1ef7fe700 1 -- 172.21.15.118:0/2217684714 >> v1:172.21.15.178:6789/0 conn(0x7ff1f006f2f0 legacy=0x7ff1f00c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1ef7fe700 1 -- 172.21.15.118:0/2217684714 >> v1:172.21.15.118:6789/0 conn(0x7ff1f0153690 legacy=0x7ff1f00c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1ef7fe700 1 -- 172.21.15.118:0/2217684714 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0153d10 con 0x7ff1f00c8200 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1ef7fe700 1 --2- 172.21.15.118:0/2217684714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f00c8200 0x7ff1f00673d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009e60 tx=0x7ff1e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ded873992dde03f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1eeffd700 1 -- 172.21.15.118:0/2217684714 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000e070 con 0x7ff1f00c8200 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1eeffd700 1 -- 172.21.15.118:0/2217684714 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e000bb60 con 0x7ff1f00c8200 2024-08-18T13:40:15.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1eeffd700 1 -- 172.21.15.118:0/2217684714 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0012430 con 0x7ff1f00c8200 2024-08-18T13:40:15.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 -- 172.21.15.118:0/2217684714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f00c8200 msgr2=0x7ff1f00673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/2217684714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f00c8200 0x7ff1f00673d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009e60 tx=0x7ff1e0009780 comp rx=0 tx=0).stop 2024-08-18T13:40:15.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 -- 172.21.15.118:0/2217684714 shutdown_connections 2024-08-18T13:40:15.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/2217684714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f00c8200 0x7ff1f00673d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 -- 172.21.15.118:0/2217684714 >> 172.21.15.118:0/2217684714 conn(0x7ff1f006c020 msgr2=0x7ff1f00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:15.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 -- 172.21.15.118:0/2217684714 shutdown_connections 2024-08-18T13:40:15.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 -- 172.21.15.118:0/2217684714 wait complete. 2024-08-18T13:40:15.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 Processor -- start 2024-08-18T13:40:15.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.554+0000 7ff1f54a9700 1 -- start start 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1f006f2f0 0x7ff1f0155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1f00c8200 0x7ff1f01555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f0153690 0x7ff1f014dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f014d170 con 0x7ff1f00c8200 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f015cf50 con 0x7ff1f0153690 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f014ef40 con 0x7ff1f006f2f0 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1f00c8200 0x7ff1f01555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1f00c8200 0x7ff1f01555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51980/0 (socket says 172.21.15.118:51980) 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 -- 172.21.15.118:0/1998609105 learned_addr learned my addr 172.21.15.118:0/1998609105 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1effff700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1f006f2f0 0x7ff1f0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f4ca8700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f0153690 0x7ff1f014dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 -- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1f006f2f0 msgr2=0x7ff1f0155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1f006f2f0 0x7ff1f0155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 -- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f0153690 msgr2=0x7ff1f014dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f0153690 0x7ff1f014dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 -- 172.21.15.118:0/1998609105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0009a20 con 0x7ff1f00c8200 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1ef7fe700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1f00c8200 0x7ff1f01555c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff1f005eb30 tx=0x7ff1e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1d7fff700 1 -- 172.21.15.118:0/1998609105 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0012c00 con 0x7ff1f00c8200 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1d7fff700 1 -- 172.21.15.118:0/1998609105 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e0012d60 con 0x7ff1f00c8200 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f014cbb0 con 0x7ff1f00c8200 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1d7fff700 1 -- 172.21.15.118:0/1998609105 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00276c0 con 0x7ff1f00c8200 2024-08-18T13:40:15.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f015cbe0 con 0x7ff1f00c8200 2024-08-18T13:40:15.565 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.558+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f014ed00 con 0x7ff1f00c8200 2024-08-18T13:40:15.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.562+0000 7ff1d7fff700 1 -- 172.21.15.118:0/1998609105 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff1e0024070 con 0x7ff1f00c8200 2024-08-18T13:40:15.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.562+0000 7ff1d7fff700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff1dc060860 0x7ff1dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.562+0000 7ff1d7fff700 1 -- 172.21.15.118:0/1998609105 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6456+0+0 (secure 0 0 0) 0x7ff1e007f360 con 0x7ff1f00c8200 2024-08-18T13:40:15.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.562+0000 7ff1effff700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff1dc060860 0x7ff1dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.568 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.562+0000 7ff1effff700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff1dc060860 0x7ff1dc062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4007fd0 tx=0x7ff1e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.566+0000 7ff1d7fff700 1 -- 172.21.15.118:0/1998609105 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff1f014ed00 con 0x7ff1f00c8200 2024-08-18T13:40:15.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.742+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1f014ed00 con 0x7ff1f00c8200 2024-08-18T13:40:15.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1d7fff700 1 -- 172.21.15.118:0/1998609105 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+17568 (secure 0 0 0) 0x7ff1f014ed00 con 0x7ff1f00c8200 2024-08-18T13:40:15.749 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:15.750 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":26,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:40:15.581311+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff1dc060860 msgr2=0x7ff1dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff1dc060860 0x7ff1dc062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4007fd0 tx=0x7ff1e4009040 comp rx=0 tx=0).stop 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1f00c8200 msgr2=0x7ff1f01555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1f00c8200 0x7ff1f01555c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff1f005eb30 tx=0x7ff1e0019000 comp rx=0 tx=0).stop 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 shutdown_connections 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff1dc060860 0x7ff1dc062d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1f0153690 0x7ff1f014dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff1f00c8200 0x7ff1f01555c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 --2- 172.21.15.118:0/1998609105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1f006f2f0 0x7ff1f0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 >> 172.21.15.118:0/1998609105 conn(0x7ff1f006c020 msgr2=0x7ff1f0159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:15.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 shutdown_connections 2024-08-18T13:40:15.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.746+0000 7ff1f54a9700 1 -- 172.21.15.118:0/1998609105 wait complete. 2024-08-18T13:40:15.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T13:39:21.246221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-18T13:39:32.826757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-18T13:39:33.731815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-18T13:39:37.447514+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-18T13:40:15.764 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T13:40:15.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.870+0000 7f75849db700 1 Processor -- start 2024-08-18T13:40:15.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.870+0000 7f75849db700 1 -- start start 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7580155710 0x7f758006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7580155ae0 0x7f75800c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7580067a60 con 0x7f7580155f40 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758006a770 con 0x7f7580155ae0 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75800c0f30 con 0x7f7580155710 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7580155ae0 0x7f75800c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f7580155ae0 0x7f75800c13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51988/0 (socket says 172.21.15.118:51988) 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 -- 172.21.15.118:0/821938505 learned_addr learned my addr 172.21.15.118:0/821938505 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:15.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757ed9d700 1 --2- 172.21.15.118:0/821938505 >> v2:172.21.15.178:3300/0 conn(0x7f7580155710 0x7f758006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 -- 172.21.15.118:0/821938505 >> v2:172.21.15.178:3300/0 conn(0x7f7580155710 msgr2=0x7f758006bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:15.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 --2- 172.21.15.118:0/821938505 >> v2:172.21.15.178:3300/0 conn(0x7f7580155710 0x7f758006bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 -- 172.21.15.118:0/821938505 >> v1:172.21.15.178:6789/0 conn(0x7f7580155f40 legacy=0x7f758014bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 -- 172.21.15.118:0/821938505 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580067780 con 0x7f7580155ae0 2024-08-18T13:40:15.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757e59c700 1 --2- 172.21.15.118:0/821938505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155ae0 0x7f75800c13c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f758006d030 tx=0x7f7568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91372db13658f44 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757dd9b700 1 -- 172.21.15.118:0/821938505 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568009e70 con 0x7f7580155ae0 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757dd9b700 1 -- 172.21.15.118:0/821938505 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800b860 con 0x7f7580155ae0 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f757dd9b700 1 -- 172.21.15.118:0/821938505 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800bc00 con 0x7f7580155ae0 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- 172.21.15.118:0/821938505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155ae0 msgr2=0x7f75800c13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 --2- 172.21.15.118:0/821938505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155ae0 0x7f75800c13c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f758006d030 tx=0x7f7568009250 comp rx=0 tx=0).stop 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- 172.21.15.118:0/821938505 shutdown_connections 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 --2- 172.21.15.118:0/821938505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155ae0 0x7f75800c13c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 --2- 172.21.15.118:0/821938505 >> v2:172.21.15.178:3300/0 conn(0x7f7580155710 0x7f758006bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- 172.21.15.118:0/821938505 >> 172.21.15.118:0/821938505 conn(0x7f758006b1f0 msgr2=0x7f758006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:15.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- 172.21.15.118:0/821938505 shutdown_connections 2024-08-18T13:40:15.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.874+0000 7f75849db700 1 -- 172.21.15.118:0/821938505 wait complete. 2024-08-18T13:40:15.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 Processor -- start 2024-08-18T13:40:15.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 -- start start 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580155710 0x7f75800c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155f40 0x7f7580151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758014fdd0 0x7f7580152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758013dfe0 con 0x7f7580155f40 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758013e260 con 0x7f7580155710 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758013e540 con 0x7f758014fdd0 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155f40 0x7f7580151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155f40 0x7f7580151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51998/0 (socket says 172.21.15.118:51998) 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 -- 172.21.15.118:0/2302445520 learned_addr learned my addr 172.21.15.118:0/2302445520 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:15.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757ed9d700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580155710 0x7f75800c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757f59e700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758014fdd0 0x7f7580152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 -- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758014fdd0 msgr2=0x7f7580152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758014fdd0 0x7f7580152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 -- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580155710 msgr2=0x7f75800c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580155710 0x7f75800c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 -- 172.21.15.118:0/2302445520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680094f0 con 0x7f7580155f40 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757f59e700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758014fdd0 0x7f7580152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757ed9d700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580155710 0x7f75800c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f757e59c700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155f40 0x7f7580151ac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f758006aea0 tx=0x7f7568017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f7576ffd700 1 -- 172.21.15.118:0/2302445520 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800bbe0 con 0x7f7580155f40 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f7576ffd700 1 -- 172.21.15.118:0/2302445520 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7568011420 con 0x7f7580155f40 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758013e820 con 0x7f7580155f40 2024-08-18T13:40:15.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f7576ffd700 1 -- 172.21.15.118:0/2302445520 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680117c0 con 0x7f7580155f40 2024-08-18T13:40:15.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.878+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758013ede0 con 0x7f7580155f40 2024-08-18T13:40:15.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.882+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7580071410 con 0x7f7580155f40 2024-08-18T13:40:15.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.882+0000 7f7576ffd700 1 -- 172.21.15.118:0/2302445520 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7568011bf0 con 0x7f7580155f40 2024-08-18T13:40:15.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.882+0000 7f7576ffd700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7560060890 0x7f7560062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:15.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.886+0000 7f7576ffd700 1 -- 172.21.15.118:0/2302445520 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6659+0+0 (secure 0 0 0) 0x7f756807e830 con 0x7f7580155f40 2024-08-18T13:40:15.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.886+0000 7f757ed9d700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7560060890 0x7f7560062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:15.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.886+0000 7f757ed9d700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7560060890 0x7f7560062d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f75700076c0 tx=0x7f757000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:15.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:15.890+0000 7f7576ffd700 1 -- 172.21.15.118:0/2302445520 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f756804e8c0 con 0x7f7580155f40 2024-08-18T13:40:16.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.054+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f758006d970 con 0x7f7580155f40 2024-08-18T13:40:16.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f7576ffd700 1 -- 172.21.15.118:0/2302445520 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7568051ee0 con 0x7f7580155f40 2024-08-18T13:40:16.061 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7560060890 msgr2=0x7f7560062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7560060890 0x7f7560062d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f75700076c0 tx=0x7f757000b040 comp rx=0 tx=0).stop 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155f40 msgr2=0x7f7580151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155f40 0x7f7580151ac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f758006aea0 tx=0x7f7568017000 comp rx=0 tx=0).stop 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 shutdown_connections 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7560060890 0x7f7560062d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f758014fdd0 0x7f7580152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7580155f40 0x7f7580151ac0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.064 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 --2- 172.21.15.118:0/2302445520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7580155710 0x7f75800c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 >> 172.21.15.118:0/2302445520 conn(0x7f758006b1f0 msgr2=0x7f758006ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:16.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 shutdown_connections 2024-08-18T13:40:16.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.058+0000 7f75849db700 1 -- 172.21.15.118:0/2302445520 wait complete. 2024-08-18T13:40:16.079 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-18T13:40:16.182 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e8d5c3700 1 Processor -- start 2024-08-18T13:40:16.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e8d5c3700 1 -- start start 2024-08-18T13:40:16.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e8d5c3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e8814d6b0 0x7f0e880c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e8d5c3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0e880c4090 0x7f0e8806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e8d5c3700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e88157360 con 0x7f0e880c36d0 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e8d5c3700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e880c3eb0 con 0x7f0e880c4090 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e8d5c3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e881577f0 con 0x7f0e8814d6b0 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0e880c4090 0x7f0e8806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0e880c4090 0x7f0e8806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52004/0 (socket says 172.21.15.118:52004) 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 -- 172.21.15.118:0/3042207231 learned_addr learned my addr 172.21.15.118:0/3042207231 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e86ffd700 1 --2- 172.21.15.118:0/3042207231 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e8814d6b0 0x7f0e880c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 -- 172.21.15.118:0/3042207231 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e8814d6b0 msgr2=0x7f0e880c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 --2- 172.21.15.118:0/3042207231 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e8814d6b0 0x7f0e880c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 -- 172.21.15.118:0/3042207231 >> v1:172.21.15.178:6789/0 conn(0x7f0e880c36d0 legacy=0x7f0e880673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 -- 172.21.15.118:0/3042207231 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880c3bd0 con 0x7f0e880c4090 2024-08-18T13:40:16.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.178+0000 7f0e877fe700 1 --2- 172.21.15.118:0/3042207231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e8806b620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009f50 tx=0x7f0e80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b639a7142d2c9189 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e85ffb700 1 -- 172.21.15.118:0/3042207231 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000e070 con 0x7f0e880c4090 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e85ffb700 1 -- 172.21.15.118:0/3042207231 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000ba70 con 0x7f0e880c4090 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e85ffb700 1 -- 172.21.15.118:0/3042207231 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800092e0 con 0x7f0e880c4090 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/3042207231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 msgr2=0x7f0e8806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/3042207231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e8806b620 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009f50 tx=0x7f0e80009840 comp rx=0 tx=0).stop 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/3042207231 shutdown_connections 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/3042207231 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e8806b620 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/3042207231 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e8814d6b0 0x7f0e880c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/3042207231 >> 172.21.15.118:0/3042207231 conn(0x7f0e880c7c40 msgr2=0x7f0e880c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:16.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/3042207231 shutdown_connections 2024-08-18T13:40:16.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.182+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/3042207231 wait complete. 2024-08-18T13:40:16.189 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 Processor -- start 2024-08-18T13:40:16.189 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 -- start start 2024-08-18T13:40:16.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e880c36d0 0x7f0e88163960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e880c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e8814d6b0 0x7f0e880bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88151590 con 0x7f0e880c4090 2024-08-18T13:40:16.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880c1520 con 0x7f0e8814d6b0 2024-08-18T13:40:16.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88166f50 con 0x7f0e880c36d0 2024-08-18T13:40:16.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e880c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e880c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52008/0 (socket says 172.21.15.118:52008) 2024-08-18T13:40:16.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 -- 172.21.15.118:0/1448644423 learned_addr learned my addr 172.21.15.118:0/1448644423 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:16.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e877fe700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e8814d6b0 0x7f0e880bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e86ffd700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e880c36d0 0x7f0e88163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 -- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e880c36d0 msgr2=0x7f0e88163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e880c36d0 0x7f0e88163960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 -- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e8814d6b0 msgr2=0x7f0e880bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e8814d6b0 0x7f0e880bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 -- 172.21.15.118:0/1448644423 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80009ae0 con 0x7f0e880c4090 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e867fc700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e880c5c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c5c0 tx=0x7f0e7800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e6effd700 1 -- 172.21.15.118:0/1448644423 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78012630 con 0x7f0e880c4090 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e6effd700 1 -- 172.21.15.118:0/1448644423 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e780127b0 con 0x7f0e880c4090 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e881671f0 con 0x7f0e880c4090 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e6effd700 1 -- 172.21.15.118:0/1448644423 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78012b70 con 0x7f0e880c4090 2024-08-18T13:40:16.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.186+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e880c1240 con 0x7f0e880c4090 2024-08-18T13:40:16.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.190+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e880642f0 con 0x7f0e880c4090 2024-08-18T13:40:16.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.190+0000 7f0e6effd700 1 -- 172.21.15.118:0/1448644423 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0e7801b400 con 0x7f0e880c4090 2024-08-18T13:40:16.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.190+0000 7f0e6effd700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0e74060860 0x7f0e74062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.190+0000 7f0e6effd700 1 -- 172.21.15.118:0/1448644423 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6659+0+0 (secure 0 0 0) 0x7f0e7807cb60 con 0x7f0e880c4090 2024-08-18T13:40:16.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.190+0000 7f0e86ffd700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0e74060860 0x7f0e74062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.197 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.190+0000 7f0e86ffd700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0e74060860 0x7f0e74062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c006fd0 tx=0x7f0e7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.198+0000 7f0e6effd700 1 -- 172.21.15.118:0/1448644423 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0e7804cbf0 con 0x7f0e880c4090 2024-08-18T13:40:16.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e880c2f70 con 0x7f0e880c4090 2024-08-18T13:40:16.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e6effd700 1 -- 172.21.15.118:0/1448644423 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e78050210 con 0x7f0e880c4090 2024-08-18T13:40:16.363 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:16.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0e74060860 msgr2=0x7f0e74062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0e74060860 0x7f0e74062d10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c006fd0 tx=0x7f0e7c008040 comp rx=0 tx=0).stop 2024-08-18T13:40:16.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 msgr2=0x7f0e880c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e880c5c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c5c0 tx=0x7f0e7800c980 comp rx=0 tx=0).stop 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 shutdown_connections 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0e74060860 0x7f0e74062d10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e8814d6b0 0x7f0e880bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.358+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0e880c4090 0x7f0e880c5c70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.362+0000 7f0e8d5c3700 1 --2- 172.21.15.118:0/1448644423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e880c36d0 0x7f0e88163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.362+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 >> 172.21.15.118:0/1448644423 conn(0x7f0e880c7c40 msgr2=0x7f0e88068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.362+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 shutdown_connections 2024-08-18T13:40:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.362+0000 7f0e8d5c3700 1 -- 172.21.15.118:0/1448644423 wait complete. 2024-08-18T13:40:16.377 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-18T13:40:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.478+0000 7f5c0b59e700 1 Processor -- start 2024-08-18T13:40:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.478+0000 7f5c0b59e700 1 -- start start 2024-08-18T13:40:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c0c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5c0c0c8d40 0x7f5c0c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c0c7980 con 0x7f5c0c0c0b00 2024-08-18T13:40:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0c2490 con 0x7f5c0c0c8d40 2024-08-18T13:40:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c156280 con 0x7f5c0c068820 2024-08-18T13:40:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5c0c0c8d40 0x7f5c0c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5c0c0c8d40 0x7f5c0c14eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52024/0 (socket says 172.21.15.118:52024) 2024-08-18T13:40:16.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 -- 172.21.15.118:0/166131928 learned_addr learned my addr 172.21.15.118:0/166131928 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:16.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c09d9b700 1 --2- 172.21.15.118:0/166131928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c0c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 -- 172.21.15.118:0/166131928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 msgr2=0x7f5c0c0c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:16.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 --2- 172.21.15.118:0/166131928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c0c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 -- 172.21.15.118:0/166131928 >> v1:172.21.15.178:6789/0 conn(0x7f5c0c0c0b00 legacy=0x7f5c0c0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 -- 172.21.15.118:0/166131928 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c155ce0 con 0x7f5c0c0c8d40 2024-08-18T13:40:16.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0ad9d700 1 --2- 172.21.15.118:0/166131928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c8d40 0x7f5c0c14eb00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009f20 tx=0x7f5c04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce24af355c4669e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0959a700 1 -- 172.21.15.118:0/166131928 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400e070 con 0x7f5c0c0c8d40 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0959a700 1 -- 172.21.15.118:0/166131928 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400b9d0 con 0x7f5c0c0c8d40 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0959a700 1 -- 172.21.15.118:0/166131928 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040092e0 con 0x7f5c0c0c8d40 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- 172.21.15.118:0/166131928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c8d40 msgr2=0x7f5c0c14eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/166131928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c8d40 0x7f5c0c14eb00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009f20 tx=0x7f5c04009840 comp rx=0 tx=0).stop 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- 172.21.15.118:0/166131928 shutdown_connections 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/166131928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c8d40 0x7f5c0c14eb00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/166131928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c0c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- 172.21.15.118:0/166131928 >> 172.21.15.118:0/166131928 conn(0x7f5c0c0cc630 msgr2=0x7f5c0c0c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- 172.21.15.118:0/166131928 shutdown_connections 2024-08-18T13:40:16.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.482+0000 7f5c0b59e700 1 -- 172.21.15.118:0/166131928 wait complete. 2024-08-18T13:40:16.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 Processor -- start 2024-08-18T13:40:16.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 -- start start 2024-08-18T13:40:16.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c0b00 0x7f5c0c151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c0c0c8d40 0x7f5c0c147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0af730 con 0x7f5c0c0c0b00 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b02b0 con 0x7f5c0c0c8d40 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0b59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0b1600 con 0x7f5c0c068820 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c09d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c0b00 0x7f5c0c151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54262/0 (socket says 172.21.15.118:54262) 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 -- 172.21.15.118:0/860160237 learned_addr learned my addr 172.21.15.118:0/860160237 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c09d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c0b00 0x7f5c0c151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52032/0 (socket says 172.21.15.118:52032) 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0ad9d700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c0c0c8d40 0x7f5c0c147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 -- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c0c0c8d40 msgr2=0x7f5c0c147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c0c0c8d40 0x7f5c0c147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 -- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c0b00 msgr2=0x7f5c0c151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c0b00 0x7f5c0c151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c0a59c700 1 -- 172.21.15.118:0/860160237 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009ae0 con 0x7f5c0c068820 2024-08-18T13:40:16.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.486+0000 7f5c09d9b700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c0b00 0x7f5c0c151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:40:16.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5c0a59c700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c154730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000c790 tx=0x7f5c00007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5bfa7fc700 1 -- 172.21.15.118:0/860160237 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00011590 con 0x7f5c0c068820 2024-08-18T13:40:16.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5bfa7fc700 1 -- 172.21.15.118:0/860160237 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00011710 con 0x7f5c0c068820 2024-08-18T13:40:16.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c169350 con 0x7f5c0c068820 2024-08-18T13:40:16.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5bfa7fc700 1 -- 172.21.15.118:0/860160237 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00011ad0 con 0x7f5c0c068820 2024-08-18T13:40:16.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c0aff40 con 0x7f5c0c068820 2024-08-18T13:40:16.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c071410 con 0x7f5c0c068820 2024-08-18T13:40:16.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5bfa7fc700 1 -- 172.21.15.118:0/860160237 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5c0001b3b0 con 0x7f5c0c068820 2024-08-18T13:40:16.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5bfa7fc700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bec060860 0x7f5bec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5bfa7fc700 1 -- 172.21.15.118:0/860160237 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6659+0+0 (secure 0 0 0) 0x7f5c0007c420 con 0x7f5c0c068820 2024-08-18T13:40:16.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5c09d9b700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bec060860 0x7f5bec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.490+0000 7f5c09d9b700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bec060860 0x7f5bec062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c067da0 tx=0x7f5bfc0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.502+0000 7f5bfa7fc700 1 -- 172.21.15.118:0/860160237 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5c0004c4b0 con 0x7f5c0c068820 2024-08-18T13:40:16.682 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.678+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c0c0c61f0 con 0x7f5c0c068820 2024-08-18T13:40:16.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.678+0000 7f5bfa7fc700 1 -- 172.21.15.118:0/860160237 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c000170a0 con 0x7f5c0c068820 2024-08-18T13:40:16.684 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 64 2024-08-18T13:40:16.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bec060860 msgr2=0x7f5bec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bec060860 0x7f5bec062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c067da0 tx=0x7f5bfc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 msgr2=0x7f5c0c154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c154730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000c790 tx=0x7f5c00007f40 comp rx=0 tx=0).stop 2024-08-18T13:40:16.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 shutdown_connections 2024-08-18T13:40:16.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5bec060860 0x7f5bec062d10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c0c0c8d40 0x7f5c0c147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5c0c0c0b00 0x7f5c0c151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 --2- 172.21.15.118:0/860160237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c0c068820 0x7f5c0c154730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 >> 172.21.15.118:0/860160237 conn(0x7f5c0c0cc630 msgr2=0x7f5c0c0c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:16.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 shutdown_connections 2024-08-18T13:40:16.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.682+0000 7f5c0b59e700 1 -- 172.21.15.118:0/860160237 wait complete. 2024-08-18T13:40:16.697 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-18T13:40:16.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.798+0000 7f0ddf59e700 1 Processor -- start 2024-08-18T13:40:16.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.798+0000 7f0ddf59e700 1 -- start start 2024-08-18T13:40:16.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c87c0 0x7f0de0068e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0de01551a0 0x7f0de00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de00689e0 con 0x7f0de00c8b90 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de0156190 con 0x7f0de01551a0 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de0155630 con 0x7f0de00c87c0 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dded9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f0de00c8b90 0x7f0de00c0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53942/0 (socket says 172.21.15.118:53942) 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dded9d700 1 -- 172.21.15.118:0/2635077576 learned_addr learned my addr 172.21.15.118:0/2635077576 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dde59c700 1 --2- 172.21.15.118:0/2635077576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c87c0 0x7f0de0068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dddd9b700 1 --2- 172.21.15.118:0/2635077576 >> v2:172.21.15.178:3300/0 conn(0x7f0de01551a0 0x7f0de00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddd59a700 1 -- 172.21.15.118:0/2635077576 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287549428 0 0) 0x7f0de00689e0 con 0x7f0de00c8b90 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddd59a700 1 -- 172.21.15.118:0/2635077576 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc8003570 con 0x7f0de00c8b90 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dde59c700 1 -- 172.21.15.118:0/2635077576 >> v2:172.21.15.178:3300/0 conn(0x7f0de01551a0 msgr2=0x7f0de00693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dde59c700 1 --2- 172.21.15.118:0/2635077576 >> v2:172.21.15.178:3300/0 conn(0x7f0de01551a0 0x7f0de00693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dde59c700 1 -- 172.21.15.118:0/2635077576 >> v1:172.21.15.118:6789/0 conn(0x7f0de00c8b90 legacy=0x7f0de00c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dde59c700 1 -- 172.21.15.118:0/2635077576 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de01558f0 con 0x7f0de00c87c0 2024-08-18T13:40:16.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0dde59c700 1 --2- 172.21.15.118:0/2635077576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c87c0 0x7f0de0068e70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40098e0 tx=0x7f0dd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cdcd93d5b02aa21 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddd59a700 1 -- 172.21.15.118:0/2635077576 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4011630 con 0x7f0de00c87c0 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddd59a700 1 -- 172.21.15.118:0/2635077576 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd4011790 con 0x7f0de00c87c0 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddd59a700 1 -- 172.21.15.118:0/2635077576 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4011b10 con 0x7f0de00c87c0 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- 172.21.15.118:0/2635077576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c87c0 msgr2=0x7f0de0068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/2635077576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c87c0 0x7f0de0068e70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40098e0 tx=0x7f0dd40091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- 172.21.15.118:0/2635077576 shutdown_connections 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/2635077576 >> v2:172.21.15.178:3300/0 conn(0x7f0de01551a0 0x7f0de00693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/2635077576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c87c0 0x7f0de0068e70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- 172.21.15.118:0/2635077576 >> 172.21.15.118:0/2635077576 conn(0x7f0de00673d0 msgr2=0x7f0de00677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- 172.21.15.118:0/2635077576 shutdown_connections 2024-08-18T13:40:16.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.802+0000 7f0ddf59e700 1 -- 172.21.15.118:0/2635077576 wait complete. 2024-08-18T13:40:16.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 Processor -- start 2024-08-18T13:40:16.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 -- start start 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de00c87c0 0x7f0de0165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c8b90 0x7f0de00c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0de01551a0 0x7f0de0148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de00cc2d0 con 0x7f0de01551a0 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de00c9970 con 0x7f0de00c87c0 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0ddf59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de01655a0 con 0x7f0de00c8b90 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0de01551a0 0x7f0de0148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dddd9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c8b90 0x7f0de00c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dde59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de00c87c0 0x7f0de0165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dddd9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c8b90 0x7f0de00c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:54286/0 (socket says 172.21.15.118:54286) 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0de01551a0 0x7f0de0148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52040/0 (socket says 172.21.15.118:52040) 2024-08-18T13:40:16.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dddd9b700 1 -- 172.21.15.118:0/1048632004 learned_addr learned my addr 172.21.15.118:0/1048632004 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 -- 172.21.15.118:0/1048632004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c8b90 msgr2=0x7f0de00c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c8b90 0x7f0de00c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 -- 172.21.15.118:0/1048632004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de00c87c0 msgr2=0x7f0de0165a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de00c87c0 0x7f0de0165a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 -- 172.21.15.118:0/1048632004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd4009420 con 0x7f0de01551a0 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dded9d700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0de01551a0 0x7f0de0148380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800cc40 tx=0x7f0dd8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.806+0000 7f0dce7fc700 1 -- 172.21.15.118:0/1048632004 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800a870 con 0x7f0de01551a0 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0dce7fc700 1 -- 172.21.15.118:0/1048632004 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd800a9d0 con 0x7f0de01551a0 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de00b0ac0 con 0x7f0de01551a0 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0dce7fc700 1 -- 172.21.15.118:0/1048632004 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800ad90 con 0x7f0de01551a0 2024-08-18T13:40:16.814 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de00b07e0 con 0x7f0de01551a0 2024-08-18T13:40:16.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de0069c10 con 0x7f0de01551a0 2024-08-18T13:40:16.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0dce7fc700 1 -- 172.21.15.118:0/1048632004 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0dd8007300 con 0x7f0de01551a0 2024-08-18T13:40:16.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0dce7fc700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dc0060860 0x7f0dc0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:16.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0dde59c700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dc0060860 0x7f0dc0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:16.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.810+0000 7f0dce7fc700 1 -- 172.21.15.118:0/1048632004 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6659+0+0 (secure 0 0 0) 0x7f0dd807d1c0 con 0x7f0de01551a0 2024-08-18T13:40:16.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.814+0000 7f0dde59c700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dc0060860 0x7f0dc0062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40091e0 tx=0x7f0dd4017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:16.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.818+0000 7f0dce7fc700 1 -- 172.21.15.118:0/1048632004 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0de0069c10 con 0x7f0de01551a0 2024-08-18T13:40:16.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.982+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0de0069c10 con 0x7f0de01551a0 2024-08-18T13:40:16.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.982+0000 7f0dce7fc700 1 -- 172.21.15.118:0/1048632004 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0de0069c10 con 0x7f0de01551a0 2024-08-18T13:40:16.988 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-08-18T13:40:16.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dc0060860 msgr2=0x7f0dc0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dc0060860 0x7f0dc0062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40091e0 tx=0x7f0dd4017000 comp rx=0 tx=0).stop 2024-08-18T13:40:16.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0de01551a0 msgr2=0x7f0de0148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0de01551a0 0x7f0de0148380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800cc40 tx=0x7f0dd8001a00 comp rx=0 tx=0).stop 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 shutdown_connections 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dc0060860 0x7f0dc0062d10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0de01551a0 0x7f0de0148380 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0de00c8b90 0x7f0de00c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 --2- 172.21.15.118:0/1048632004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0de00c87c0 0x7f0de0165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 >> 172.21.15.118:0/1048632004 conn(0x7f0de00673d0 msgr2=0x7f0de014b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 shutdown_connections 2024-08-18T13:40:16.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:16.986+0000 7f0ddf59e700 1 -- 172.21.15.118:0/1048632004 wait complete. 2024-08-18T13:40:17.002 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:40:17.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.098+0000 7eff0172c700 1 Processor -- start 2024-08-18T13:40:17.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- start start 2024-08-18T13:40:17.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc05e6b0 0x7efefc152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efefc0c0c50 0x7efefc0c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc0c1610 con 0x7efefc14bf60 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc068370 con 0x7efefc0c0c50 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc152d80 con 0x7efefc05e6b0 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efefc0c0c50 0x7efefc0c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7efefc0c0c50 0x7efefc0c10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52052/0 (socket says 172.21.15.118:52052) 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 -- 172.21.15.118:0/164816018 learned_addr learned my addr 172.21.15.118:0/164816018 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7efef3fff700 1 --2- 172.21.15.118:0/164816018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc05e6b0 0x7efefc152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 -- 172.21.15.118:0/164816018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc05e6b0 msgr2=0x7efefc152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 --2- 172.21.15.118:0/164816018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc05e6b0 0x7efefc152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 -- 172.21.15.118:0/164816018 >> v1:172.21.15.178:6789/0 conn(0x7efefc14bf60 legacy=0x7efefc06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 -- 172.21.15.118:0/164816018 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc067470 con 0x7efefc0c0c50 2024-08-18T13:40:17.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff00f2b700 1 --2- 172.21.15.118:0/164816018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc0c10d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efef400d1a0 tx=0x7efef400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a16da05bc2d7abe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7efefb7fe700 1 -- 172.21.15.118:0/164816018 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4010070 con 0x7efefc0c0c50 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7efefb7fe700 1 -- 172.21.15.118:0/164816018 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400b0d0 con 0x7efefc0c0c50 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7efefb7fe700 1 -- 172.21.15.118:0/164816018 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400dea0 con 0x7efefc0c0c50 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- 172.21.15.118:0/164816018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 msgr2=0x7efefc0c10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 --2- 172.21.15.118:0/164816018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc0c10d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efef400d1a0 tx=0x7efef400ba50 comp rx=0 tx=0).stop 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- 172.21.15.118:0/164816018 shutdown_connections 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 --2- 172.21.15.118:0/164816018 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc0c10d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 --2- 172.21.15.118:0/164816018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc05e6b0 0x7efefc152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- 172.21.15.118:0/164816018 >> 172.21.15.118:0/164816018 conn(0x7efefc0c81e0 msgr2=0x7efefc0c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- 172.21.15.118:0/164816018 shutdown_connections 2024-08-18T13:40:17.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.102+0000 7eff0172c700 1 -- 172.21.15.118:0/164816018 wait complete. 2024-08-18T13:40:17.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 Processor -- start 2024-08-18T13:40:17.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 -- start start 2024-08-18T13:40:17.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efefc05e6b0 0x7efefc154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc164020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc14bf60 0x7efefc06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc144740 con 0x7efefc0c0c50 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc147be0 con 0x7efefc05e6b0 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff0172c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1631d0 con 0x7efefc14bf60 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7efef3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc164020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7efef3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc164020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52066/0 (socket says 172.21.15.118:52066) 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7efef3fff700 1 -- 172.21.15.118:0/1729407003 learned_addr learned my addr 172.21.15.118:0/1729407003 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7eff00f2b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc14bf60 0x7efefc06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.106+0000 7efefbfff700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efefc05e6b0 0x7efefc154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef3fff700 1 -- 172.21.15.118:0/1729407003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc14bf60 msgr2=0x7efefc06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef3fff700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc14bf60 0x7efefc06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef3fff700 1 -- 172.21.15.118:0/1729407003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efefc05e6b0 msgr2=0x7efefc154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef3fff700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efefc05e6b0 0x7efefc154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef3fff700 1 -- 172.21.15.118:0/1729407003 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef400bcf0 con 0x7efefc0c0c50 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef3fff700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc164020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efee400e930 tx=0x7efee40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef8ff9700 1 -- 172.21.15.118:0/1729407003 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4013680 con 0x7efefc0c0c50 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef8ff9700 1 -- 172.21.15.118:0/1729407003 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee40137e0 con 0x7efefc0c0c50 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc1440f0 con 0x7efefc0c0c50 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef8ff9700 1 -- 172.21.15.118:0/1729407003 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4013b80 con 0x7efefc0c0c50 2024-08-18T13:40:17.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc149530 con 0x7efefc0c0c50 2024-08-18T13:40:17.116 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef8ff9700 1 -- 172.21.15.118:0/1729407003 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efee400c470 con 0x7efefc0c0c50 2024-08-18T13:40:17.116 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc1448f0 con 0x7efefc0c0c50 2024-08-18T13:40:17.116 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.110+0000 7efef8ff9700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7efedc060890 0x7efedc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.117 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.114+0000 7efef8ff9700 1 -- 172.21.15.118:0/1729407003 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6659+0+0 (secure 0 0 0) 0x7efee407d020 con 0x7efefc0c0c50 2024-08-18T13:40:17.117 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.114+0000 7efefbfff700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7efedc060890 0x7efedc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.117 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.114+0000 7efefbfff700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7efedc060890 0x7efedc062d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efeec003a80 tx=0x7efeec007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:17.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.118+0000 7efef8ff9700 1 -- 172.21.15.118:0/1729407003 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efee404d0b0 con 0x7efefc0c0c50 2024-08-18T13:40:17.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.318+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efefc0713a0 con 0x7efefc0c0c50 2024-08-18T13:40:17.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.318+0000 7efef8ff9700 1 -- 172.21.15.118:0/1729407003 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7efee40506d0 con 0x7efefc0c0c50 2024-08-18T13:40:17.324 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:17.324 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:40:17.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7efedc060890 msgr2=0x7efedc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7efedc060890 0x7efedc062d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efeec003a80 tx=0x7efeec007ec0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 msgr2=0x7efefc164020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc164020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efee400e930 tx=0x7efee40091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 shutdown_connections 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7efedc060890 0x7efedc062d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efefc14bf60 0x7efefc06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efefc0c0c50 0x7efefc164020 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 --2- 172.21.15.118:0/1729407003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efefc05e6b0 0x7efefc154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 >> 172.21.15.118:0/1729407003 conn(0x7efefc0c81e0 msgr2=0x7efefc1538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 shutdown_connections 2024-08-18T13:40:17.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.322+0000 7eff0172c700 1 -- 172.21.15.118:0/1729407003 wait complete. 2024-08-18T13:40:17.328 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:40:17.338 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:40:17.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.442+0000 7f85932e9700 1 Processor -- start 2024-08-18T13:40:17.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.442+0000 7f85932e9700 1 -- start start 2024-08-18T13:40:17.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.442+0000 7f85932e9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c153690 0x7f858c0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.442+0000 7f85932e9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f858c0c8200 0x7f858c06f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.442+0000 7f85932e9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c06ee60 con 0x7f858c0c22a0 2024-08-18T13:40:17.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.442+0000 7f85932e9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c15a640 con 0x7f858c0c8200 2024-08-18T13:40:17.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.442+0000 7f85932e9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c153d10 con 0x7f858c153690 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f858c0c8200 0x7f858c06f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85922e7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c153690 0x7f858c0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f858c0c8200 0x7f858c06f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52078/0 (socket says 172.21.15.118:52078) 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 -- 172.21.15.118:0/2302944568 learned_addr learned my addr 172.21.15.118:0/2302944568 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 -- 172.21.15.118:0/2302944568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c153690 msgr2=0x7f858c0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 --2- 172.21.15.118:0/2302944568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c153690 0x7f858c0673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 -- 172.21.15.118:0/2302944568 >> v1:172.21.15.178:6789/0 conn(0x7f858c0c22a0 legacy=0x7f858c0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 -- 172.21.15.118:0/2302944568 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c06f9c0 con 0x7f858c0c8200 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f8591ae6700 1 --2- 172.21.15.118:0/2302944568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c0c8200 0x7f858c06f2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f857c0099b0 tx=0x7f857c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73ebf3353fdcb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85912e5700 1 -- 172.21.15.118:0/2302944568 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c009ea0 con 0x7f858c0c8200 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85912e5700 1 -- 172.21.15.118:0/2302944568 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c00b860 con 0x7f858c0c8200 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85912e5700 1 -- 172.21.15.118:0/2302944568 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c00bc00 con 0x7f858c0c8200 2024-08-18T13:40:17.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 -- 172.21.15.118:0/2302944568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c0c8200 msgr2=0x7f858c06f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 --2- 172.21.15.118:0/2302944568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c0c8200 0x7f858c06f2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f857c0099b0 tx=0x7f857c009250 comp rx=0 tx=0).stop 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 -- 172.21.15.118:0/2302944568 shutdown_connections 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 --2- 172.21.15.118:0/2302944568 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c0c8200 0x7f858c06f2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 --2- 172.21.15.118:0/2302944568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c153690 0x7f858c0673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 -- 172.21.15.118:0/2302944568 >> 172.21.15.118:0/2302944568 conn(0x7f858c06c020 msgr2=0x7f858c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 -- 172.21.15.118:0/2302944568 shutdown_connections 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.446+0000 7f85932e9700 1 -- 172.21.15.118:0/2302944568 wait complete. 2024-08-18T13:40:17.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 Processor -- start 2024-08-18T13:40:17.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 -- start start 2024-08-18T13:40:17.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c0c22a0 0x7f858c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f858c0c8200 0x7f858c0c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c153690 0x7f858c169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c157080 con 0x7f858c153690 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c156470 con 0x7f858c0c8200 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c156750 con 0x7f858c0c22a0 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c153690 0x7f858c169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c153690 0x7f858c169e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52094/0 (socket says 172.21.15.118:52094) 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85922e7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c0c22a0 0x7f858c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 -- 172.21.15.118:0/2124469215 learned_addr learned my addr 172.21.15.118:0/2124469215 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:40:17.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8591ae6700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f858c0c8200 0x7f858c0c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 -- 172.21.15.118:0/2124469215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c0c22a0 msgr2=0x7f858c0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c0c22a0 0x7f858c0c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 -- 172.21.15.118:0/2124469215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f858c0c8200 msgr2=0x7f858c0c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f858c0c8200 0x7f858c0c01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 -- 172.21.15.118:0/2124469215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c0094f0 con 0x7f858c153690 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f8592ae8700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c153690 0x7f858c169e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f858400c8e0 tx=0x7f8584001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85827fc700 1 -- 172.21.15.118:0/2124469215 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584012630 con 0x7f858c153690 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85827fc700 1 -- 172.21.15.118:0/2124469215 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8584012790 con 0x7f858c153690 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c156a30 con 0x7f858c153690 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85827fc700 1 -- 172.21.15.118:0/2124469215 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584012b50 con 0x7f858c153690 2024-08-18T13:40:17.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.450+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c0c5790 con 0x7f858c153690 2024-08-18T13:40:17.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.454+0000 7f85827fc700 1 -- 172.21.15.118:0/2124469215 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f858401a3a0 con 0x7f858c153690 2024-08-18T13:40:17.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.454+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858c05ecf0 con 0x7f858c153690 2024-08-18T13:40:17.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.454+0000 7f85827fc700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8570060890 0x7f8570062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:40:17.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.454+0000 7f85827fc700 1 -- 172.21.15.118:0/2124469215 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6659+0+0 (secure 0 0 0) 0x7f858407d240 con 0x7f858c153690 2024-08-18T13:40:17.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.454+0000 7f85922e7700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8570060890 0x7f8570062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:40:17.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.454+0000 7f85922e7700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8570060890 0x7f8570062d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8588009d10 tx=0x7f85880091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:40:17.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.462+0000 7f85827fc700 1 -- 172.21.15.118:0/2124469215 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f858c05ecf0 con 0x7f858c153690 2024-08-18T13:40:17.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.626+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f858c0c2000 con 0x7f858c153690 2024-08-18T13:40:17.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85827fc700 1 -- 172.21.15.118:0/2124469215 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+17568 (secure 0 0 0) 0x7f858c0c2000 con 0x7f858c153690 2024-08-18T13:40:17.633 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:40:17.634 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":27,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:40:16.588772+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:40:17.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8570060890 msgr2=0x7f8570062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8570060890 0x7f8570062d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8588009d10 tx=0x7f85880091a0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c153690 msgr2=0x7f858c169e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c153690 0x7f858c169e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f858400c8e0 tx=0x7f8584001710 comp rx=0 tx=0).stop 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 shutdown_connections 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8570060890 0x7f8570062d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f858c153690 0x7f858c169e70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f858c0c8200 0x7f858c0c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 --2- 172.21.15.118:0/2124469215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f858c0c22a0 0x7f858c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 >> 172.21.15.118:0/2124469215 conn(0x7f858c06c020 msgr2=0x7f858c0c2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 shutdown_connections 2024-08-18T13:40:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:40:17.630+0000 7f85932e9700 1 -- 172.21.15.118:0/2124469215 wait complete. 2024-08-18T13:40:17.653 DEBUG:tasks.mds_thrash:Yielding 2024-08-18T13:40:17.654 INFO:teuthology.run_tasks:Running task workunit... 2024-08-18T13:40:17.668 INFO:tasks.workunit:Pulling workunits from ref a17c67c052c1476b20f99d847823755b26703128 2024-08-18T13:40:17.669 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-18T13:40:17.669 INFO:tasks.workunit:timeout=3h 2024-08-18T13:40:17.669 INFO:tasks.workunit:cleanup=True 2024-08-18T13:40:17.669 DEBUG:teuthology.orchestra.run.smithi118:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:17.670 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2024-08-18T13:40:17.670 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 104 secs before thrashing 2024-08-18T13:40:17.677 INFO:teuthology.orchestra.run.smithi118.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:17.678 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-08-18T13:40:17.678 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 34h/52d Inode: 1 Links: 2 2024-08-18T13:40:17.678 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T13:40:17.678 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-08-18T13:40:17.678 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-08-18 13:39:40.415378037 +0000 2024-08-18T13:40:17.678 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-08-18 13:39:48.059376436 +0000 2024-08-18T13:40:17.678 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2024-08-18T13:40:17.678 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-18T13:40:17.678 DEBUG:teuthology.orchestra.run.smithi118:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-18T13:40:17.747 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a17c67c052c1476b20f99d847823755b26703128 2024-08-18T13:40:17.755 INFO:tasks.workunit.client.0.smithi118.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-18T13:40:41.457 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:40:41.460 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:41:11.514 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:41:11.517 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:41:18.552 INFO:tasks.workunit.client.0.smithi118.stderr:Note: switching to 'a17c67c052c1476b20f99d847823755b26703128'. 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr:state without impacting any branches by switching back to a branch. 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr: git switch -c 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr:Or undo this operation with: 2024-08-18T13:41:18.553 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-08-18T13:41:18.554 INFO:tasks.workunit.client.0.smithi118.stderr: git switch - 2024-08-18T13:41:18.554 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-08-18T13:41:18.554 INFO:tasks.workunit.client.0.smithi118.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-18T13:41:18.554 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-08-18T13:41:18.554 INFO:tasks.workunit.client.0.smithi118.stderr:HEAD is now at a17c67c052c mgr/volumes: print thread name on lock 2024-08-18T13:41:18.558 DEBUG:teuthology.orchestra.run.smithi118:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-18T13:41:18.589 INFO:tasks.workunit.client.0.smithi118.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-18T13:41:18.592 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-18T13:41:18.592 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-18T13:41:19.405 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-18T13:41:19.554 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-18T13:41:19.613 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-18T13:41:19.615 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-18T13:41:19.615 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-18T13:41:19.675 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-18T13:41:19.681 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-08-18T13:41:19.682 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-18T13:41:19.688 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2024-08-18T13:41:19.689 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-08-18T13:41:19.690 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-08-18T13:41:19.713 INFO:tasks.workunit.client.0.smithi118.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-08-18T13:41:24.294 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir foo 2024-08-18T13:41:24.311 INFO:tasks.workunit.client.0.smithi118.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-08-18T13:41:25.090 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir foo/.snap/barsnap 2024-08-18T13:41:25.112 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm foo/linux-2.6.33.tar.bz2 2024-08-18T13:41:25.129 INFO:tasks.workunit.client.0.smithi118.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2024-08-18T13:41:25.282 INFO:tasks.workunit.client.0.smithi118.stdout:passed: files are identical 2024-08-18T13:41:25.282 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo passed: files are identical 2024-08-18T13:41:25.282 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir foo/.snap/barsnap 2024-08-18T13:41:25.302 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-08-18T13:41:25.303 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:41:25.303 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:41:25.304 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:41:25.407 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2024-08-18T13:41:25.407 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2024-08-18T13:41:25.431 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 1 2024-08-18T13:41:25.435 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 2 2024-08-18T13:41:25.438 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 3 2024-08-18T13:41:25.442 INFO:tasks.workunit.client.0.smithi118.stderr:+ '[' -e file4 ']' 2024-08-18T13:41:25.442 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir .snap/snap1 2024-08-18T13:41:25.459 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 4 2024-08-18T13:41:25.474 INFO:tasks.workunit.client.0.smithi118.stderr:++ ls 2024-08-18T13:41:25.482 INFO:tasks.workunit.client.0.smithi118.stderr:+ now='file1 2024-08-18T13:41:25.482 INFO:tasks.workunit.client.0.smithi118.stderr:file2 2024-08-18T13:41:25.482 INFO:tasks.workunit.client.0.smithi118.stderr:file3 2024-08-18T13:41:25.483 INFO:tasks.workunit.client.0.smithi118.stderr:file4' 2024-08-18T13:41:25.483 INFO:tasks.workunit.client.0.smithi118.stderr:++ ls .snap/snap1 2024-08-18T13:41:25.494 INFO:tasks.workunit.client.0.smithi118.stderr:+ then='file1 2024-08-18T13:41:25.494 INFO:tasks.workunit.client.0.smithi118.stderr:file2 2024-08-18T13:41:25.494 INFO:tasks.workunit.client.0.smithi118.stderr:file3' 2024-08-18T13:41:25.494 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir .snap/snap1 2024-08-18T13:41:25.509 INFO:tasks.workunit.client.0.smithi118.stderr:+ '[' 'file1 2024-08-18T13:41:25.509 INFO:tasks.workunit.client.0.smithi118.stderr:file2 2024-08-18T13:41:25.509 INFO:tasks.workunit.client.0.smithi118.stderr:file3 2024-08-18T13:41:25.509 INFO:tasks.workunit.client.0.smithi118.stderr:file4' = 'file1 2024-08-18T13:41:25.509 INFO:tasks.workunit.client.0.smithi118.stderr:file2 2024-08-18T13:41:25.509 INFO:tasks.workunit.client.0.smithi118.stderr:file3' ']' 2024-08-18T13:41:25.509 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 1 2024-08-18T13:41:25.520 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 2 2024-08-18T13:41:25.534 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 3 2024-08-18T13:41:25.542 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir .snap/snap1 2024-08-18T13:41:25.555 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo 4 2024-08-18T13:41:25.569 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir .snap/snap1 2024-08-18T13:41:25.585 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm file1 file2 file3 file4 2024-08-18T13:41:25.599 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-08-18T13:41:25.600 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:41:25.600 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:41:25.600 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:41:25.640 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2024-08-18T13:41:25.641 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2024-08-18T13:41:25.710 INFO:tasks.workunit.client.0.smithi118.stdout:Create dir 100 to 199 ... 2024-08-18T13:41:25.722 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 100 2024-08-18T13:41:25.787 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 101 2024-08-18T13:41:25.818 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 102 2024-08-18T13:41:25.851 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 103 2024-08-18T13:41:25.885 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 104 2024-08-18T13:41:25.918 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 105 2024-08-18T13:41:25.953 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 106 2024-08-18T13:41:25.986 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 107 2024-08-18T13:41:26.017 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 108 2024-08-18T13:41:26.052 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 109 2024-08-18T13:41:26.085 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 110 2024-08-18T13:41:26.120 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 111 2024-08-18T13:41:26.154 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 112 2024-08-18T13:41:26.189 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 113 2024-08-18T13:41:26.224 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 114 2024-08-18T13:41:26.258 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 115 2024-08-18T13:41:26.291 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 116 2024-08-18T13:41:26.324 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 117 2024-08-18T13:41:26.357 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 118 2024-08-18T13:41:26.391 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 119 2024-08-18T13:41:26.424 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 120 2024-08-18T13:41:26.457 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 121 2024-08-18T13:41:26.496 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 122 2024-08-18T13:41:26.548 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 123 2024-08-18T13:41:26.587 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 124 2024-08-18T13:41:26.619 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 125 2024-08-18T13:41:26.651 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 126 2024-08-18T13:41:26.685 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 127 2024-08-18T13:41:26.723 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 128 2024-08-18T13:41:26.775 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 129 2024-08-18T13:41:26.820 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 130 2024-08-18T13:41:26.850 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 131 2024-08-18T13:41:26.881 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 132 2024-08-18T13:41:26.915 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 133 2024-08-18T13:41:26.948 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 134 2024-08-18T13:41:26.981 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 135 2024-08-18T13:41:27.014 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 136 2024-08-18T13:41:27.050 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 137 2024-08-18T13:41:27.084 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 138 2024-08-18T13:41:27.120 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 139 2024-08-18T13:41:27.158 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 140 2024-08-18T13:41:27.192 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 141 2024-08-18T13:41:27.227 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 142 2024-08-18T13:41:27.261 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 143 2024-08-18T13:41:27.296 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 144 2024-08-18T13:41:27.332 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 145 2024-08-18T13:41:27.365 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 146 2024-08-18T13:41:27.400 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 147 2024-08-18T13:41:27.473 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 148 2024-08-18T13:41:27.473 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 149 2024-08-18T13:41:27.509 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 150 2024-08-18T13:41:27.542 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 151 2024-08-18T13:41:27.582 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 152 2024-08-18T13:41:27.617 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 153 2024-08-18T13:41:27.651 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 154 2024-08-18T13:41:27.686 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 155 2024-08-18T13:41:27.720 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 156 2024-08-18T13:41:27.753 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 157 2024-08-18T13:41:27.785 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 158 2024-08-18T13:41:27.816 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 159 2024-08-18T13:41:27.849 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 160 2024-08-18T13:41:27.884 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 161 2024-08-18T13:41:27.919 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 162 2024-08-18T13:41:27.953 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 163 2024-08-18T13:41:27.989 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 164 2024-08-18T13:41:28.021 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 165 2024-08-18T13:41:28.055 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 166 2024-08-18T13:41:28.087 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 167 2024-08-18T13:41:28.123 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 168 2024-08-18T13:41:28.160 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 169 2024-08-18T13:41:28.195 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 170 2024-08-18T13:41:28.233 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 171 2024-08-18T13:41:28.267 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 172 2024-08-18T13:41:28.301 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 173 2024-08-18T13:41:28.333 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 174 2024-08-18T13:41:28.368 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 175 2024-08-18T13:41:28.403 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 176 2024-08-18T13:41:28.438 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 177 2024-08-18T13:41:28.474 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 178 2024-08-18T13:41:28.515 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 179 2024-08-18T13:41:28.551 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 180 2024-08-18T13:41:28.587 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 181 2024-08-18T13:41:28.622 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 182 2024-08-18T13:41:28.914 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 183 2024-08-18T13:41:28.950 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 184 2024-08-18T13:41:28.985 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 185 2024-08-18T13:41:29.022 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 186 2024-08-18T13:41:29.059 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 187 2024-08-18T13:41:29.096 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 188 2024-08-18T13:41:29.133 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 189 2024-08-18T13:41:29.168 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 190 2024-08-18T13:41:29.204 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 191 2024-08-18T13:41:29.238 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 192 2024-08-18T13:41:29.273 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 193 2024-08-18T13:41:29.307 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 194 2024-08-18T13:41:29.340 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 195 2024-08-18T13:41:29.377 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 196 2024-08-18T13:41:29.413 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 197 2024-08-18T13:41:29.447 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 198 2024-08-18T13:41:29.482 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 199 2024-08-18T13:41:29.536 INFO:tasks.workunit.client.0.smithi118.stdout:Take first snapshot .snap/test1 2024-08-18T13:41:29.690 INFO:tasks.workunit.client.0.smithi118.stdout:Create dir 200 to 299 ... 2024-08-18T13:41:29.691 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 200 2024-08-18T13:41:30.057 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 201 2024-08-18T13:41:30.090 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 202 2024-08-18T13:41:30.124 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 203 2024-08-18T13:41:30.158 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 204 2024-08-18T13:41:30.191 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 205 2024-08-18T13:41:30.224 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 206 2024-08-18T13:41:30.254 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 207 2024-08-18T13:41:30.285 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 208 2024-08-18T13:41:30.320 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 209 2024-08-18T13:41:30.353 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 210 2024-08-18T13:41:30.385 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 211 2024-08-18T13:41:30.415 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 212 2024-08-18T13:41:30.448 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 213 2024-08-18T13:41:30.487 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 214 2024-08-18T13:41:30.533 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 215 2024-08-18T13:41:30.563 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 216 2024-08-18T13:41:30.596 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 217 2024-08-18T13:41:30.628 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 218 2024-08-18T13:41:30.665 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 219 2024-08-18T13:41:30.697 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 220 2024-08-18T13:41:30.728 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 221 2024-08-18T13:41:30.761 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 222 2024-08-18T13:41:30.793 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 223 2024-08-18T13:41:30.825 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 224 2024-08-18T13:41:30.857 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 225 2024-08-18T13:41:30.888 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 226 2024-08-18T13:41:30.918 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 227 2024-08-18T13:41:30.950 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 228 2024-08-18T13:41:30.982 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 229 2024-08-18T13:41:31.012 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 230 2024-08-18T13:41:31.043 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 231 2024-08-18T13:41:31.076 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 232 2024-08-18T13:41:31.109 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 233 2024-08-18T13:41:31.142 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 234 2024-08-18T13:41:31.177 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 235 2024-08-18T13:41:31.209 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 236 2024-08-18T13:41:31.240 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 237 2024-08-18T13:41:31.272 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 238 2024-08-18T13:41:31.306 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 239 2024-08-18T13:41:31.338 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 240 2024-08-18T13:41:31.369 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 241 2024-08-18T13:41:31.402 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 242 2024-08-18T13:41:31.436 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 243 2024-08-18T13:41:31.471 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 244 2024-08-18T13:41:31.502 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 245 2024-08-18T13:41:31.642 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 246 2024-08-18T13:41:31.674 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 247 2024-08-18T13:41:31.705 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 248 2024-08-18T13:41:31.735 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 249 2024-08-18T13:41:31.766 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 250 2024-08-18T13:41:31.806 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 251 2024-08-18T13:41:34.796 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 252 2024-08-18T13:41:34.828 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 253 2024-08-18T13:41:34.861 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 254 2024-08-18T13:41:34.897 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 255 2024-08-18T13:41:34.932 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 256 2024-08-18T13:41:34.965 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 257 2024-08-18T13:41:35.001 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 258 2024-08-18T13:41:35.034 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 259 2024-08-18T13:41:35.068 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 260 2024-08-18T13:41:35.101 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 261 2024-08-18T13:41:35.131 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 262 2024-08-18T13:41:35.161 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 263 2024-08-18T13:41:35.195 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 264 2024-08-18T13:41:35.226 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 265 2024-08-18T13:41:35.259 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 266 2024-08-18T13:41:35.290 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 267 2024-08-18T13:41:35.324 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 268 2024-08-18T13:41:35.359 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 269 2024-08-18T13:41:35.394 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 270 2024-08-18T13:41:35.428 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 271 2024-08-18T13:41:35.462 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 272 2024-08-18T13:41:35.578 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 273 2024-08-18T13:41:35.680 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 274 2024-08-18T13:41:35.715 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 275 2024-08-18T13:41:35.750 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 276 2024-08-18T13:41:35.784 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 277 2024-08-18T13:41:35.818 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 278 2024-08-18T13:41:35.850 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 279 2024-08-18T13:41:35.883 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 280 2024-08-18T13:41:35.917 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 281 2024-08-18T13:41:35.949 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 282 2024-08-18T13:41:35.982 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 283 2024-08-18T13:41:36.015 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 284 2024-08-18T13:41:36.046 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 285 2024-08-18T13:41:36.078 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 286 2024-08-18T13:41:36.110 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 287 2024-08-18T13:41:36.143 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 288 2024-08-18T13:41:36.177 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 289 2024-08-18T13:41:36.208 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 290 2024-08-18T13:41:36.241 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 291 2024-08-18T13:41:36.275 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 292 2024-08-18T13:41:36.307 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 293 2024-08-18T13:41:36.341 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 294 2024-08-18T13:41:36.373 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 295 2024-08-18T13:41:36.406 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 296 2024-08-18T13:41:36.438 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 297 2024-08-18T13:41:36.469 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 298 2024-08-18T13:41:36.501 INFO:tasks.workunit.client.0.smithi118.stdout: create dir 299 2024-08-18T13:41:36.531 INFO:tasks.workunit.client.0.smithi118.stdout:Create a snapshot in every first level dir ... 2024-08-18T13:41:36.857 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/.snap/snap-subdir-test 2024-08-18T13:41:36.941 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/30 file after the snapshot 2024-08-18T13:41:36.946 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/31 file after the snapshot 2024-08-18T13:41:36.951 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/32 file after the snapshot 2024-08-18T13:41:36.956 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/33 file after the snapshot 2024-08-18T13:41:36.959 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/34 file after the snapshot 2024-08-18T13:41:36.961 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/35 file after the snapshot 2024-08-18T13:41:36.965 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/36 file after the snapshot 2024-08-18T13:41:36.968 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/37 file after the snapshot 2024-08-18T13:41:36.971 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/38 file after the snapshot 2024-08-18T13:41:36.973 INFO:tasks.workunit.client.0.smithi118.stdout: create 100/39 file after the snapshot 2024-08-18T13:41:36.976 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/.snap/snap-subdir-test 2024-08-18T13:41:37.007 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/30 file after the snapshot 2024-08-18T13:41:37.011 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/31 file after the snapshot 2024-08-18T13:41:37.016 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/32 file after the snapshot 2024-08-18T13:41:37.020 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/33 file after the snapshot 2024-08-18T13:41:37.025 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/34 file after the snapshot 2024-08-18T13:41:37.028 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/35 file after the snapshot 2024-08-18T13:41:37.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/36 file after the snapshot 2024-08-18T13:41:37.034 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/37 file after the snapshot 2024-08-18T13:41:37.037 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/38 file after the snapshot 2024-08-18T13:41:37.040 INFO:tasks.workunit.client.0.smithi118.stdout: create 101/39 file after the snapshot 2024-08-18T13:41:37.043 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/.snap/snap-subdir-test 2024-08-18T13:41:37.072 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/30 file after the snapshot 2024-08-18T13:41:37.076 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/31 file after the snapshot 2024-08-18T13:41:37.080 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/32 file after the snapshot 2024-08-18T13:41:37.085 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/33 file after the snapshot 2024-08-18T13:41:37.089 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/34 file after the snapshot 2024-08-18T13:41:37.092 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/35 file after the snapshot 2024-08-18T13:41:37.094 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/36 file after the snapshot 2024-08-18T13:41:37.097 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/37 file after the snapshot 2024-08-18T13:41:37.100 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/38 file after the snapshot 2024-08-18T13:41:37.103 INFO:tasks.workunit.client.0.smithi118.stdout: create 102/39 file after the snapshot 2024-08-18T13:41:37.105 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/.snap/snap-subdir-test 2024-08-18T13:41:37.138 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/30 file after the snapshot 2024-08-18T13:41:37.142 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/31 file after the snapshot 2024-08-18T13:41:37.146 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/32 file after the snapshot 2024-08-18T13:41:37.151 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/33 file after the snapshot 2024-08-18T13:41:37.155 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/34 file after the snapshot 2024-08-18T13:41:37.158 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/35 file after the snapshot 2024-08-18T13:41:37.162 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/36 file after the snapshot 2024-08-18T13:41:37.165 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/37 file after the snapshot 2024-08-18T13:41:37.168 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/38 file after the snapshot 2024-08-18T13:41:37.172 INFO:tasks.workunit.client.0.smithi118.stdout: create 103/39 file after the snapshot 2024-08-18T13:41:37.174 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/.snap/snap-subdir-test 2024-08-18T13:41:37.206 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/30 file after the snapshot 2024-08-18T13:41:37.210 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/31 file after the snapshot 2024-08-18T13:41:37.216 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/32 file after the snapshot 2024-08-18T13:41:37.221 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/33 file after the snapshot 2024-08-18T13:41:37.225 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/34 file after the snapshot 2024-08-18T13:41:37.228 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/35 file after the snapshot 2024-08-18T13:41:37.231 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/36 file after the snapshot 2024-08-18T13:41:37.233 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/37 file after the snapshot 2024-08-18T13:41:37.236 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/38 file after the snapshot 2024-08-18T13:41:37.239 INFO:tasks.workunit.client.0.smithi118.stdout: create 104/39 file after the snapshot 2024-08-18T13:41:37.242 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/.snap/snap-subdir-test 2024-08-18T13:41:37.273 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/30 file after the snapshot 2024-08-18T13:41:37.277 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/31 file after the snapshot 2024-08-18T13:41:37.282 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/32 file after the snapshot 2024-08-18T13:41:37.287 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/33 file after the snapshot 2024-08-18T13:41:37.290 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/34 file after the snapshot 2024-08-18T13:41:37.293 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/35 file after the snapshot 2024-08-18T13:41:37.296 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/36 file after the snapshot 2024-08-18T13:41:37.300 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/37 file after the snapshot 2024-08-18T13:41:37.303 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/38 file after the snapshot 2024-08-18T13:41:37.306 INFO:tasks.workunit.client.0.smithi118.stdout: create 105/39 file after the snapshot 2024-08-18T13:41:37.309 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/.snap/snap-subdir-test 2024-08-18T13:41:37.351 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/30 file after the snapshot 2024-08-18T13:41:37.355 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/31 file after the snapshot 2024-08-18T13:41:37.361 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/32 file after the snapshot 2024-08-18T13:41:37.366 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/33 file after the snapshot 2024-08-18T13:41:37.369 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/34 file after the snapshot 2024-08-18T13:41:37.373 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/35 file after the snapshot 2024-08-18T13:41:37.376 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/36 file after the snapshot 2024-08-18T13:41:37.380 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/37 file after the snapshot 2024-08-18T13:41:37.383 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/38 file after the snapshot 2024-08-18T13:41:37.386 INFO:tasks.workunit.client.0.smithi118.stdout: create 106/39 file after the snapshot 2024-08-18T13:41:37.389 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/.snap/snap-subdir-test 2024-08-18T13:41:37.420 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/30 file after the snapshot 2024-08-18T13:41:37.424 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/31 file after the snapshot 2024-08-18T13:41:37.429 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/32 file after the snapshot 2024-08-18T13:41:37.434 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/33 file after the snapshot 2024-08-18T13:41:37.438 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/34 file after the snapshot 2024-08-18T13:41:37.441 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/35 file after the snapshot 2024-08-18T13:41:37.444 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/36 file after the snapshot 2024-08-18T13:41:37.447 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/37 file after the snapshot 2024-08-18T13:41:37.450 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/38 file after the snapshot 2024-08-18T13:41:37.453 INFO:tasks.workunit.client.0.smithi118.stdout: create 107/39 file after the snapshot 2024-08-18T13:41:37.456 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/.snap/snap-subdir-test 2024-08-18T13:41:37.488 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/30 file after the snapshot 2024-08-18T13:41:37.491 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/31 file after the snapshot 2024-08-18T13:41:37.496 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/32 file after the snapshot 2024-08-18T13:41:37.501 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/33 file after the snapshot 2024-08-18T13:41:37.504 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/34 file after the snapshot 2024-08-18T13:41:37.508 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/35 file after the snapshot 2024-08-18T13:41:37.512 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/36 file after the snapshot 2024-08-18T13:41:37.515 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/37 file after the snapshot 2024-08-18T13:41:37.518 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/38 file after the snapshot 2024-08-18T13:41:37.521 INFO:tasks.workunit.client.0.smithi118.stdout: create 108/39 file after the snapshot 2024-08-18T13:41:37.525 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/.snap/snap-subdir-test 2024-08-18T13:41:37.557 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/30 file after the snapshot 2024-08-18T13:41:37.562 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/31 file after the snapshot 2024-08-18T13:41:37.566 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/32 file after the snapshot 2024-08-18T13:41:37.570 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/33 file after the snapshot 2024-08-18T13:41:37.574 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/34 file after the snapshot 2024-08-18T13:41:37.577 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/35 file after the snapshot 2024-08-18T13:41:37.580 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/36 file after the snapshot 2024-08-18T13:41:37.583 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/37 file after the snapshot 2024-08-18T13:41:37.587 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/38 file after the snapshot 2024-08-18T13:41:37.590 INFO:tasks.workunit.client.0.smithi118.stdout: create 109/39 file after the snapshot 2024-08-18T13:41:37.592 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/.snap/snap-subdir-test 2024-08-18T13:41:37.630 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/30 file after the snapshot 2024-08-18T13:41:37.661 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/31 file after the snapshot 2024-08-18T13:41:37.664 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/32 file after the snapshot 2024-08-18T13:41:37.668 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/33 file after the snapshot 2024-08-18T13:41:37.671 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/34 file after the snapshot 2024-08-18T13:41:37.673 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/35 file after the snapshot 2024-08-18T13:41:37.677 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/36 file after the snapshot 2024-08-18T13:41:37.679 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/37 file after the snapshot 2024-08-18T13:41:37.682 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/38 file after the snapshot 2024-08-18T13:41:37.684 INFO:tasks.workunit.client.0.smithi118.stdout: create 110/39 file after the snapshot 2024-08-18T13:41:37.687 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/.snap/snap-subdir-test 2024-08-18T13:41:37.744 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/30 file after the snapshot 2024-08-18T13:41:37.749 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/31 file after the snapshot 2024-08-18T13:41:37.754 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/32 file after the snapshot 2024-08-18T13:41:37.757 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/33 file after the snapshot 2024-08-18T13:41:37.762 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/34 file after the snapshot 2024-08-18T13:41:37.764 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/35 file after the snapshot 2024-08-18T13:41:37.768 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/36 file after the snapshot 2024-08-18T13:41:37.771 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/37 file after the snapshot 2024-08-18T13:41:37.774 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/38 file after the snapshot 2024-08-18T13:41:37.777 INFO:tasks.workunit.client.0.smithi118.stdout: create 111/39 file after the snapshot 2024-08-18T13:41:37.780 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/.snap/snap-subdir-test 2024-08-18T13:41:37.814 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/30 file after the snapshot 2024-08-18T13:41:37.818 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/31 file after the snapshot 2024-08-18T13:41:37.822 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/32 file after the snapshot 2024-08-18T13:41:37.826 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/33 file after the snapshot 2024-08-18T13:41:37.829 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/34 file after the snapshot 2024-08-18T13:41:37.831 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/35 file after the snapshot 2024-08-18T13:41:37.834 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/36 file after the snapshot 2024-08-18T13:41:37.838 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/37 file after the snapshot 2024-08-18T13:41:37.839 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/38 file after the snapshot 2024-08-18T13:41:37.843 INFO:tasks.workunit.client.0.smithi118.stdout: create 112/39 file after the snapshot 2024-08-18T13:41:37.845 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/.snap/snap-subdir-test 2024-08-18T13:41:37.878 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/30 file after the snapshot 2024-08-18T13:41:37.882 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/31 file after the snapshot 2024-08-18T13:41:37.887 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/32 file after the snapshot 2024-08-18T13:41:37.892 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/33 file after the snapshot 2024-08-18T13:41:37.896 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/34 file after the snapshot 2024-08-18T13:41:37.900 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/35 file after the snapshot 2024-08-18T13:41:37.902 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/36 file after the snapshot 2024-08-18T13:41:37.905 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/37 file after the snapshot 2024-08-18T13:41:37.908 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/38 file after the snapshot 2024-08-18T13:41:37.910 INFO:tasks.workunit.client.0.smithi118.stdout: create 113/39 file after the snapshot 2024-08-18T13:41:37.914 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/.snap/snap-subdir-test 2024-08-18T13:41:37.950 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/30 file after the snapshot 2024-08-18T13:41:37.954 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/31 file after the snapshot 2024-08-18T13:41:37.959 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/32 file after the snapshot 2024-08-18T13:41:37.963 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/33 file after the snapshot 2024-08-18T13:41:37.967 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/34 file after the snapshot 2024-08-18T13:41:37.971 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/35 file after the snapshot 2024-08-18T13:41:37.973 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/36 file after the snapshot 2024-08-18T13:41:37.976 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/37 file after the snapshot 2024-08-18T13:41:37.979 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/38 file after the snapshot 2024-08-18T13:41:37.985 INFO:tasks.workunit.client.0.smithi118.stdout: create 114/39 file after the snapshot 2024-08-18T13:41:37.986 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/.snap/snap-subdir-test 2024-08-18T13:41:38.016 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/30 file after the snapshot 2024-08-18T13:41:38.019 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/31 file after the snapshot 2024-08-18T13:41:38.024 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/32 file after the snapshot 2024-08-18T13:41:38.028 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/33 file after the snapshot 2024-08-18T13:41:38.032 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/34 file after the snapshot 2024-08-18T13:41:38.035 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/35 file after the snapshot 2024-08-18T13:41:38.037 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/36 file after the snapshot 2024-08-18T13:41:38.040 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/37 file after the snapshot 2024-08-18T13:41:38.043 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/38 file after the snapshot 2024-08-18T13:41:38.045 INFO:tasks.workunit.client.0.smithi118.stdout: create 115/39 file after the snapshot 2024-08-18T13:41:38.049 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/.snap/snap-subdir-test 2024-08-18T13:41:38.078 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/30 file after the snapshot 2024-08-18T13:41:38.082 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/31 file after the snapshot 2024-08-18T13:41:38.087 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/32 file after the snapshot 2024-08-18T13:41:38.092 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/33 file after the snapshot 2024-08-18T13:41:38.095 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/34 file after the snapshot 2024-08-18T13:41:38.099 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/35 file after the snapshot 2024-08-18T13:41:38.102 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/36 file after the snapshot 2024-08-18T13:41:38.105 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/37 file after the snapshot 2024-08-18T13:41:38.108 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/38 file after the snapshot 2024-08-18T13:41:38.111 INFO:tasks.workunit.client.0.smithi118.stdout: create 116/39 file after the snapshot 2024-08-18T13:41:38.114 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/.snap/snap-subdir-test 2024-08-18T13:41:38.147 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/30 file after the snapshot 2024-08-18T13:41:38.152 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/31 file after the snapshot 2024-08-18T13:41:38.156 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/32 file after the snapshot 2024-08-18T13:41:38.162 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/33 file after the snapshot 2024-08-18T13:41:38.166 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/34 file after the snapshot 2024-08-18T13:41:38.168 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/35 file after the snapshot 2024-08-18T13:41:38.172 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/36 file after the snapshot 2024-08-18T13:41:38.175 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/37 file after the snapshot 2024-08-18T13:41:38.179 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/38 file after the snapshot 2024-08-18T13:41:38.182 INFO:tasks.workunit.client.0.smithi118.stdout: create 117/39 file after the snapshot 2024-08-18T13:41:38.185 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/.snap/snap-subdir-test 2024-08-18T13:41:38.216 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/30 file after the snapshot 2024-08-18T13:41:38.220 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/31 file after the snapshot 2024-08-18T13:41:38.224 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/32 file after the snapshot 2024-08-18T13:41:38.228 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/33 file after the snapshot 2024-08-18T13:41:38.232 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/34 file after the snapshot 2024-08-18T13:41:38.235 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/35 file after the snapshot 2024-08-18T13:41:38.238 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/36 file after the snapshot 2024-08-18T13:41:38.241 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/37 file after the snapshot 2024-08-18T13:41:38.244 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/38 file after the snapshot 2024-08-18T13:41:38.247 INFO:tasks.workunit.client.0.smithi118.stdout: create 118/39 file after the snapshot 2024-08-18T13:41:38.250 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/.snap/snap-subdir-test 2024-08-18T13:41:38.283 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/30 file after the snapshot 2024-08-18T13:41:38.287 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/31 file after the snapshot 2024-08-18T13:41:38.291 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/32 file after the snapshot 2024-08-18T13:41:38.297 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/33 file after the snapshot 2024-08-18T13:41:38.300 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/34 file after the snapshot 2024-08-18T13:41:38.304 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/35 file after the snapshot 2024-08-18T13:41:38.307 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/36 file after the snapshot 2024-08-18T13:41:38.311 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/37 file after the snapshot 2024-08-18T13:41:38.314 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/38 file after the snapshot 2024-08-18T13:41:38.317 INFO:tasks.workunit.client.0.smithi118.stdout: create 119/39 file after the snapshot 2024-08-18T13:41:38.320 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/.snap/snap-subdir-test 2024-08-18T13:41:38.353 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/30 file after the snapshot 2024-08-18T13:41:38.356 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/31 file after the snapshot 2024-08-18T13:41:38.360 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/32 file after the snapshot 2024-08-18T13:41:38.363 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/33 file after the snapshot 2024-08-18T13:41:38.368 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/34 file after the snapshot 2024-08-18T13:41:38.371 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/35 file after the snapshot 2024-08-18T13:41:38.374 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/36 file after the snapshot 2024-08-18T13:41:38.377 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/37 file after the snapshot 2024-08-18T13:41:38.380 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/38 file after the snapshot 2024-08-18T13:41:38.382 INFO:tasks.workunit.client.0.smithi118.stdout: create 120/39 file after the snapshot 2024-08-18T13:41:38.386 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/.snap/snap-subdir-test 2024-08-18T13:41:38.417 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/30 file after the snapshot 2024-08-18T13:41:38.421 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/31 file after the snapshot 2024-08-18T13:41:38.425 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/32 file after the snapshot 2024-08-18T13:41:38.430 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/33 file after the snapshot 2024-08-18T13:41:38.433 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/34 file after the snapshot 2024-08-18T13:41:38.437 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/35 file after the snapshot 2024-08-18T13:41:38.440 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/36 file after the snapshot 2024-08-18T13:41:38.443 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/37 file after the snapshot 2024-08-18T13:41:38.445 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/38 file after the snapshot 2024-08-18T13:41:38.449 INFO:tasks.workunit.client.0.smithi118.stdout: create 121/39 file after the snapshot 2024-08-18T13:41:38.451 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/.snap/snap-subdir-test 2024-08-18T13:41:38.489 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/30 file after the snapshot 2024-08-18T13:41:38.493 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/31 file after the snapshot 2024-08-18T13:41:38.497 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/32 file after the snapshot 2024-08-18T13:41:38.500 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/33 file after the snapshot 2024-08-18T13:41:38.504 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/34 file after the snapshot 2024-08-18T13:41:38.508 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/35 file after the snapshot 2024-08-18T13:41:38.512 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/36 file after the snapshot 2024-08-18T13:41:38.515 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/37 file after the snapshot 2024-08-18T13:41:38.517 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/38 file after the snapshot 2024-08-18T13:41:38.521 INFO:tasks.workunit.client.0.smithi118.stdout: create 122/39 file after the snapshot 2024-08-18T13:41:38.524 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/.snap/snap-subdir-test 2024-08-18T13:41:38.558 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/30 file after the snapshot 2024-08-18T13:41:38.562 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/31 file after the snapshot 2024-08-18T13:41:38.567 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/32 file after the snapshot 2024-08-18T13:41:38.571 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/33 file after the snapshot 2024-08-18T13:41:38.574 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/34 file after the snapshot 2024-08-18T13:41:38.578 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/35 file after the snapshot 2024-08-18T13:41:38.581 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/36 file after the snapshot 2024-08-18T13:41:38.584 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/37 file after the snapshot 2024-08-18T13:41:38.587 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/38 file after the snapshot 2024-08-18T13:41:38.590 INFO:tasks.workunit.client.0.smithi118.stdout: create 123/39 file after the snapshot 2024-08-18T13:41:38.593 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/.snap/snap-subdir-test 2024-08-18T13:41:38.626 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/30 file after the snapshot 2024-08-18T13:41:38.630 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/31 file after the snapshot 2024-08-18T13:41:38.634 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/32 file after the snapshot 2024-08-18T13:41:38.639 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/33 file after the snapshot 2024-08-18T13:41:38.645 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/34 file after the snapshot 2024-08-18T13:41:38.649 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/35 file after the snapshot 2024-08-18T13:41:38.652 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/36 file after the snapshot 2024-08-18T13:41:38.656 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/37 file after the snapshot 2024-08-18T13:41:38.663 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/38 file after the snapshot 2024-08-18T13:41:38.663 INFO:tasks.workunit.client.0.smithi118.stdout: create 124/39 file after the snapshot 2024-08-18T13:41:38.666 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/.snap/snap-subdir-test 2024-08-18T13:41:38.698 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/30 file after the snapshot 2024-08-18T13:41:38.702 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/31 file after the snapshot 2024-08-18T13:41:38.705 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/32 file after the snapshot 2024-08-18T13:41:38.710 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/33 file after the snapshot 2024-08-18T13:41:38.713 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/34 file after the snapshot 2024-08-18T13:41:38.715 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/35 file after the snapshot 2024-08-18T13:41:38.718 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/36 file after the snapshot 2024-08-18T13:41:38.720 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/37 file after the snapshot 2024-08-18T13:41:38.723 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/38 file after the snapshot 2024-08-18T13:41:38.726 INFO:tasks.workunit.client.0.smithi118.stdout: create 125/39 file after the snapshot 2024-08-18T13:41:38.729 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/.snap/snap-subdir-test 2024-08-18T13:41:38.757 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/30 file after the snapshot 2024-08-18T13:41:38.761 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/31 file after the snapshot 2024-08-18T13:41:38.765 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/32 file after the snapshot 2024-08-18T13:41:38.770 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/33 file after the snapshot 2024-08-18T13:41:38.774 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/34 file after the snapshot 2024-08-18T13:41:38.777 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/35 file after the snapshot 2024-08-18T13:41:38.780 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/36 file after the snapshot 2024-08-18T13:41:38.784 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/37 file after the snapshot 2024-08-18T13:41:38.786 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/38 file after the snapshot 2024-08-18T13:41:38.789 INFO:tasks.workunit.client.0.smithi118.stdout: create 126/39 file after the snapshot 2024-08-18T13:41:38.792 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/.snap/snap-subdir-test 2024-08-18T13:41:38.824 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/30 file after the snapshot 2024-08-18T13:41:38.827 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/31 file after the snapshot 2024-08-18T13:41:38.831 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/32 file after the snapshot 2024-08-18T13:41:38.835 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/33 file after the snapshot 2024-08-18T13:41:38.838 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/34 file after the snapshot 2024-08-18T13:41:38.841 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/35 file after the snapshot 2024-08-18T13:41:38.844 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/36 file after the snapshot 2024-08-18T13:41:38.847 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/37 file after the snapshot 2024-08-18T13:41:38.851 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/38 file after the snapshot 2024-08-18T13:41:38.854 INFO:tasks.workunit.client.0.smithi118.stdout: create 127/39 file after the snapshot 2024-08-18T13:41:38.856 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/.snap/snap-subdir-test 2024-08-18T13:41:38.890 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/30 file after the snapshot 2024-08-18T13:41:38.894 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/31 file after the snapshot 2024-08-18T13:41:38.898 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/32 file after the snapshot 2024-08-18T13:41:38.903 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/33 file after the snapshot 2024-08-18T13:41:38.907 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/34 file after the snapshot 2024-08-18T13:41:38.910 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/35 file after the snapshot 2024-08-18T13:41:38.913 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/36 file after the snapshot 2024-08-18T13:41:38.916 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/37 file after the snapshot 2024-08-18T13:41:38.919 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/38 file after the snapshot 2024-08-18T13:41:38.922 INFO:tasks.workunit.client.0.smithi118.stdout: create 128/39 file after the snapshot 2024-08-18T13:41:38.925 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/.snap/snap-subdir-test 2024-08-18T13:41:38.955 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/30 file after the snapshot 2024-08-18T13:41:38.959 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/31 file after the snapshot 2024-08-18T13:41:38.963 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/32 file after the snapshot 2024-08-18T13:41:38.967 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/33 file after the snapshot 2024-08-18T13:41:38.970 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/34 file after the snapshot 2024-08-18T13:41:38.975 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/35 file after the snapshot 2024-08-18T13:41:38.977 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/36 file after the snapshot 2024-08-18T13:41:38.980 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/37 file after the snapshot 2024-08-18T13:41:38.983 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/38 file after the snapshot 2024-08-18T13:41:38.986 INFO:tasks.workunit.client.0.smithi118.stdout: create 129/39 file after the snapshot 2024-08-18T13:41:38.989 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/.snap/snap-subdir-test 2024-08-18T13:41:39.020 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/30 file after the snapshot 2024-08-18T13:41:39.024 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/31 file after the snapshot 2024-08-18T13:41:39.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/32 file after the snapshot 2024-08-18T13:41:39.033 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/33 file after the snapshot 2024-08-18T13:41:39.037 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/34 file after the snapshot 2024-08-18T13:41:39.041 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/35 file after the snapshot 2024-08-18T13:41:39.044 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/36 file after the snapshot 2024-08-18T13:41:39.047 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/37 file after the snapshot 2024-08-18T13:41:39.051 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/38 file after the snapshot 2024-08-18T13:41:39.054 INFO:tasks.workunit.client.0.smithi118.stdout: create 130/39 file after the snapshot 2024-08-18T13:41:39.057 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/.snap/snap-subdir-test 2024-08-18T13:41:39.089 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/30 file after the snapshot 2024-08-18T13:41:39.094 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/31 file after the snapshot 2024-08-18T13:41:39.098 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/32 file after the snapshot 2024-08-18T13:41:39.102 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/33 file after the snapshot 2024-08-18T13:41:39.105 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/34 file after the snapshot 2024-08-18T13:41:39.109 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/35 file after the snapshot 2024-08-18T13:41:39.112 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/36 file after the snapshot 2024-08-18T13:41:39.115 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/37 file after the snapshot 2024-08-18T13:41:39.118 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/38 file after the snapshot 2024-08-18T13:41:39.121 INFO:tasks.workunit.client.0.smithi118.stdout: create 131/39 file after the snapshot 2024-08-18T13:41:39.124 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/.snap/snap-subdir-test 2024-08-18T13:41:39.156 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/30 file after the snapshot 2024-08-18T13:41:39.159 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/31 file after the snapshot 2024-08-18T13:41:39.163 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/32 file after the snapshot 2024-08-18T13:41:39.167 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/33 file after the snapshot 2024-08-18T13:41:39.171 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/34 file after the snapshot 2024-08-18T13:41:39.174 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/35 file after the snapshot 2024-08-18T13:41:39.176 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/36 file after the snapshot 2024-08-18T13:41:39.180 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/37 file after the snapshot 2024-08-18T13:41:39.182 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/38 file after the snapshot 2024-08-18T13:41:39.185 INFO:tasks.workunit.client.0.smithi118.stdout: create 132/39 file after the snapshot 2024-08-18T13:41:39.188 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/.snap/snap-subdir-test 2024-08-18T13:41:39.220 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/30 file after the snapshot 2024-08-18T13:41:39.224 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/31 file after the snapshot 2024-08-18T13:41:39.227 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/32 file after the snapshot 2024-08-18T13:41:39.231 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/33 file after the snapshot 2024-08-18T13:41:39.236 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/34 file after the snapshot 2024-08-18T13:41:39.238 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/35 file after the snapshot 2024-08-18T13:41:39.241 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/36 file after the snapshot 2024-08-18T13:41:39.244 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/37 file after the snapshot 2024-08-18T13:41:39.247 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/38 file after the snapshot 2024-08-18T13:41:39.249 INFO:tasks.workunit.client.0.smithi118.stdout: create 133/39 file after the snapshot 2024-08-18T13:41:39.252 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/.snap/snap-subdir-test 2024-08-18T13:41:39.283 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/30 file after the snapshot 2024-08-18T13:41:39.286 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/31 file after the snapshot 2024-08-18T13:41:39.291 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/32 file after the snapshot 2024-08-18T13:41:39.296 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/33 file after the snapshot 2024-08-18T13:41:39.300 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/34 file after the snapshot 2024-08-18T13:41:39.304 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/35 file after the snapshot 2024-08-18T13:41:39.308 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/36 file after the snapshot 2024-08-18T13:41:39.311 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/37 file after the snapshot 2024-08-18T13:41:39.314 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/38 file after the snapshot 2024-08-18T13:41:39.317 INFO:tasks.workunit.client.0.smithi118.stdout: create 134/39 file after the snapshot 2024-08-18T13:41:39.319 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/.snap/snap-subdir-test 2024-08-18T13:41:39.352 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/30 file after the snapshot 2024-08-18T13:41:39.356 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/31 file after the snapshot 2024-08-18T13:41:39.360 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/32 file after the snapshot 2024-08-18T13:41:39.365 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/33 file after the snapshot 2024-08-18T13:41:39.367 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/34 file after the snapshot 2024-08-18T13:41:39.373 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/35 file after the snapshot 2024-08-18T13:41:39.374 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/36 file after the snapshot 2024-08-18T13:41:39.377 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/37 file after the snapshot 2024-08-18T13:41:39.380 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/38 file after the snapshot 2024-08-18T13:41:39.383 INFO:tasks.workunit.client.0.smithi118.stdout: create 135/39 file after the snapshot 2024-08-18T13:41:39.386 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/.snap/snap-subdir-test 2024-08-18T13:41:39.420 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/30 file after the snapshot 2024-08-18T13:41:39.423 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/31 file after the snapshot 2024-08-18T13:41:39.428 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/32 file after the snapshot 2024-08-18T13:41:39.432 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/33 file after the snapshot 2024-08-18T13:41:39.435 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/34 file after the snapshot 2024-08-18T13:41:39.439 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/35 file after the snapshot 2024-08-18T13:41:39.441 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/36 file after the snapshot 2024-08-18T13:41:39.445 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/37 file after the snapshot 2024-08-18T13:41:39.449 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/38 file after the snapshot 2024-08-18T13:41:39.452 INFO:tasks.workunit.client.0.smithi118.stdout: create 136/39 file after the snapshot 2024-08-18T13:41:39.455 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/.snap/snap-subdir-test 2024-08-18T13:41:39.487 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/30 file after the snapshot 2024-08-18T13:41:39.491 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/31 file after the snapshot 2024-08-18T13:41:39.495 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/32 file after the snapshot 2024-08-18T13:41:39.499 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/33 file after the snapshot 2024-08-18T13:41:39.503 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/34 file after the snapshot 2024-08-18T13:41:39.506 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/35 file after the snapshot 2024-08-18T13:41:39.509 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/36 file after the snapshot 2024-08-18T13:41:39.512 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/37 file after the snapshot 2024-08-18T13:41:39.515 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/38 file after the snapshot 2024-08-18T13:41:39.518 INFO:tasks.workunit.client.0.smithi118.stdout: create 137/39 file after the snapshot 2024-08-18T13:41:39.521 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/.snap/snap-subdir-test 2024-08-18T13:41:39.551 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/30 file after the snapshot 2024-08-18T13:41:39.555 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/31 file after the snapshot 2024-08-18T13:41:39.560 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/32 file after the snapshot 2024-08-18T13:41:39.565 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/33 file after the snapshot 2024-08-18T13:41:39.568 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/34 file after the snapshot 2024-08-18T13:41:39.572 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/35 file after the snapshot 2024-08-18T13:41:39.576 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/36 file after the snapshot 2024-08-18T13:41:39.579 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/37 file after the snapshot 2024-08-18T13:41:39.582 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/38 file after the snapshot 2024-08-18T13:41:39.584 INFO:tasks.workunit.client.0.smithi118.stdout: create 138/39 file after the snapshot 2024-08-18T13:41:39.587 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/.snap/snap-subdir-test 2024-08-18T13:41:39.628 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/30 file after the snapshot 2024-08-18T13:41:39.632 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/31 file after the snapshot 2024-08-18T13:41:39.638 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/32 file after the snapshot 2024-08-18T13:41:39.643 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/33 file after the snapshot 2024-08-18T13:41:39.648 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/34 file after the snapshot 2024-08-18T13:41:39.651 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/35 file after the snapshot 2024-08-18T13:41:39.655 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/36 file after the snapshot 2024-08-18T13:41:39.658 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/37 file after the snapshot 2024-08-18T13:41:39.661 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/38 file after the snapshot 2024-08-18T13:41:39.665 INFO:tasks.workunit.client.0.smithi118.stdout: create 139/39 file after the snapshot 2024-08-18T13:41:39.668 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/.snap/snap-subdir-test 2024-08-18T13:41:39.702 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/30 file after the snapshot 2024-08-18T13:41:39.706 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/31 file after the snapshot 2024-08-18T13:41:39.710 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/32 file after the snapshot 2024-08-18T13:41:39.715 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/33 file after the snapshot 2024-08-18T13:41:39.718 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/34 file after the snapshot 2024-08-18T13:41:39.721 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/35 file after the snapshot 2024-08-18T13:41:39.724 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/36 file after the snapshot 2024-08-18T13:41:39.727 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/37 file after the snapshot 2024-08-18T13:41:39.731 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/38 file after the snapshot 2024-08-18T13:41:39.733 INFO:tasks.workunit.client.0.smithi118.stdout: create 140/39 file after the snapshot 2024-08-18T13:41:39.737 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/.snap/snap-subdir-test 2024-08-18T13:41:39.767 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/30 file after the snapshot 2024-08-18T13:41:39.771 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/31 file after the snapshot 2024-08-18T13:41:39.775 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/32 file after the snapshot 2024-08-18T13:41:39.780 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/33 file after the snapshot 2024-08-18T13:41:39.784 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/34 file after the snapshot 2024-08-18T13:41:39.787 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/35 file after the snapshot 2024-08-18T13:41:39.791 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/36 file after the snapshot 2024-08-18T13:41:39.794 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/37 file after the snapshot 2024-08-18T13:41:39.797 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/38 file after the snapshot 2024-08-18T13:41:39.800 INFO:tasks.workunit.client.0.smithi118.stdout: create 141/39 file after the snapshot 2024-08-18T13:41:39.804 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/.snap/snap-subdir-test 2024-08-18T13:41:39.839 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/30 file after the snapshot 2024-08-18T13:41:39.844 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/31 file after the snapshot 2024-08-18T13:41:39.849 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/32 file after the snapshot 2024-08-18T13:41:39.853 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/33 file after the snapshot 2024-08-18T13:41:39.857 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/34 file after the snapshot 2024-08-18T13:41:39.860 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/35 file after the snapshot 2024-08-18T13:41:39.864 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/36 file after the snapshot 2024-08-18T13:41:39.867 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/37 file after the snapshot 2024-08-18T13:41:39.870 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/38 file after the snapshot 2024-08-18T13:41:39.873 INFO:tasks.workunit.client.0.smithi118.stdout: create 142/39 file after the snapshot 2024-08-18T13:41:39.876 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/.snap/snap-subdir-test 2024-08-18T13:41:39.903 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/30 file after the snapshot 2024-08-18T13:41:39.907 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/31 file after the snapshot 2024-08-18T13:41:39.911 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/32 file after the snapshot 2024-08-18T13:41:39.914 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/33 file after the snapshot 2024-08-18T13:41:39.918 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/34 file after the snapshot 2024-08-18T13:41:39.921 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/35 file after the snapshot 2024-08-18T13:41:39.926 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/36 file after the snapshot 2024-08-18T13:41:39.928 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/37 file after the snapshot 2024-08-18T13:41:39.932 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/38 file after the snapshot 2024-08-18T13:41:39.936 INFO:tasks.workunit.client.0.smithi118.stdout: create 143/39 file after the snapshot 2024-08-18T13:41:39.939 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/.snap/snap-subdir-test 2024-08-18T13:41:39.970 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/30 file after the snapshot 2024-08-18T13:41:39.974 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/31 file after the snapshot 2024-08-18T13:41:39.978 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/32 file after the snapshot 2024-08-18T13:41:39.983 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/33 file after the snapshot 2024-08-18T13:41:39.987 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/34 file after the snapshot 2024-08-18T13:41:39.990 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/35 file after the snapshot 2024-08-18T13:41:39.993 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/36 file after the snapshot 2024-08-18T13:41:39.996 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/37 file after the snapshot 2024-08-18T13:41:40.002 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/38 file after the snapshot 2024-08-18T13:41:40.003 INFO:tasks.workunit.client.0.smithi118.stdout: create 144/39 file after the snapshot 2024-08-18T13:41:40.006 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/.snap/snap-subdir-test 2024-08-18T13:41:40.036 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/30 file after the snapshot 2024-08-18T13:41:40.040 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/31 file after the snapshot 2024-08-18T13:41:40.045 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/32 file after the snapshot 2024-08-18T13:41:40.048 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/33 file after the snapshot 2024-08-18T13:41:40.053 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/34 file after the snapshot 2024-08-18T13:41:40.056 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/35 file after the snapshot 2024-08-18T13:41:40.059 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/36 file after the snapshot 2024-08-18T13:41:40.062 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/37 file after the snapshot 2024-08-18T13:41:40.065 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/38 file after the snapshot 2024-08-18T13:41:40.068 INFO:tasks.workunit.client.0.smithi118.stdout: create 145/39 file after the snapshot 2024-08-18T13:41:40.071 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/.snap/snap-subdir-test 2024-08-18T13:41:40.103 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/30 file after the snapshot 2024-08-18T13:41:40.107 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/31 file after the snapshot 2024-08-18T13:41:40.111 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/32 file after the snapshot 2024-08-18T13:41:40.115 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/33 file after the snapshot 2024-08-18T13:41:40.118 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/34 file after the snapshot 2024-08-18T13:41:40.122 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/35 file after the snapshot 2024-08-18T13:41:40.125 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/36 file after the snapshot 2024-08-18T13:41:40.129 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/37 file after the snapshot 2024-08-18T13:41:40.131 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/38 file after the snapshot 2024-08-18T13:41:40.134 INFO:tasks.workunit.client.0.smithi118.stdout: create 146/39 file after the snapshot 2024-08-18T13:41:40.137 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/.snap/snap-subdir-test 2024-08-18T13:41:40.168 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/30 file after the snapshot 2024-08-18T13:41:40.172 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/31 file after the snapshot 2024-08-18T13:41:40.176 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/32 file after the snapshot 2024-08-18T13:41:40.181 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/33 file after the snapshot 2024-08-18T13:41:40.186 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/34 file after the snapshot 2024-08-18T13:41:40.188 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/35 file after the snapshot 2024-08-18T13:41:40.191 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/36 file after the snapshot 2024-08-18T13:41:40.194 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/37 file after the snapshot 2024-08-18T13:41:40.198 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/38 file after the snapshot 2024-08-18T13:41:40.201 INFO:tasks.workunit.client.0.smithi118.stdout: create 147/39 file after the snapshot 2024-08-18T13:41:40.204 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/.snap/snap-subdir-test 2024-08-18T13:41:40.237 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/30 file after the snapshot 2024-08-18T13:41:40.240 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/31 file after the snapshot 2024-08-18T13:41:40.243 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/32 file after the snapshot 2024-08-18T13:41:40.248 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/33 file after the snapshot 2024-08-18T13:41:40.251 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/34 file after the snapshot 2024-08-18T13:41:40.254 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/35 file after the snapshot 2024-08-18T13:41:40.257 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/36 file after the snapshot 2024-08-18T13:41:40.260 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/37 file after the snapshot 2024-08-18T13:41:40.263 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/38 file after the snapshot 2024-08-18T13:41:40.266 INFO:tasks.workunit.client.0.smithi118.stdout: create 148/39 file after the snapshot 2024-08-18T13:41:40.269 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/.snap/snap-subdir-test 2024-08-18T13:41:40.302 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/30 file after the snapshot 2024-08-18T13:41:40.303 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/31 file after the snapshot 2024-08-18T13:41:40.307 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/32 file after the snapshot 2024-08-18T13:41:40.312 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/33 file after the snapshot 2024-08-18T13:41:40.315 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/34 file after the snapshot 2024-08-18T13:41:40.318 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/35 file after the snapshot 2024-08-18T13:41:40.321 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/36 file after the snapshot 2024-08-18T13:41:40.324 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/37 file after the snapshot 2024-08-18T13:41:40.327 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/38 file after the snapshot 2024-08-18T13:41:40.330 INFO:tasks.workunit.client.0.smithi118.stdout: create 149/39 file after the snapshot 2024-08-18T13:41:40.332 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/.snap/snap-subdir-test 2024-08-18T13:41:40.372 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/30 file after the snapshot 2024-08-18T13:41:40.377 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/31 file after the snapshot 2024-08-18T13:41:40.382 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/32 file after the snapshot 2024-08-18T13:41:40.386 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/33 file after the snapshot 2024-08-18T13:41:40.389 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/34 file after the snapshot 2024-08-18T13:41:40.393 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/35 file after the snapshot 2024-08-18T13:41:40.397 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/36 file after the snapshot 2024-08-18T13:41:40.400 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/37 file after the snapshot 2024-08-18T13:41:40.403 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/38 file after the snapshot 2024-08-18T13:41:40.406 INFO:tasks.workunit.client.0.smithi118.stdout: create 150/39 file after the snapshot 2024-08-18T13:41:40.409 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/.snap/snap-subdir-test 2024-08-18T13:41:40.450 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/30 file after the snapshot 2024-08-18T13:41:40.455 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/31 file after the snapshot 2024-08-18T13:41:40.459 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/32 file after the snapshot 2024-08-18T13:41:40.548 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/33 file after the snapshot 2024-08-18T13:41:40.551 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/34 file after the snapshot 2024-08-18T13:41:40.554 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/35 file after the snapshot 2024-08-18T13:41:40.557 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/36 file after the snapshot 2024-08-18T13:41:40.562 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/37 file after the snapshot 2024-08-18T13:41:40.565 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/38 file after the snapshot 2024-08-18T13:41:40.567 INFO:tasks.workunit.client.0.smithi118.stdout: create 151/39 file after the snapshot 2024-08-18T13:41:40.570 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/.snap/snap-subdir-test 2024-08-18T13:41:40.607 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/30 file after the snapshot 2024-08-18T13:41:40.611 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/31 file after the snapshot 2024-08-18T13:41:40.615 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/32 file after the snapshot 2024-08-18T13:41:40.620 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/33 file after the snapshot 2024-08-18T13:41:40.624 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/34 file after the snapshot 2024-08-18T13:41:40.627 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/35 file after the snapshot 2024-08-18T13:41:40.630 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/36 file after the snapshot 2024-08-18T13:41:40.633 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/37 file after the snapshot 2024-08-18T13:41:40.636 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/38 file after the snapshot 2024-08-18T13:41:40.639 INFO:tasks.workunit.client.0.smithi118.stdout: create 152/39 file after the snapshot 2024-08-18T13:41:40.642 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/.snap/snap-subdir-test 2024-08-18T13:41:40.690 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/30 file after the snapshot 2024-08-18T13:41:40.756 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/31 file after the snapshot 2024-08-18T13:41:40.760 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/32 file after the snapshot 2024-08-18T13:41:40.764 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/33 file after the snapshot 2024-08-18T13:41:40.768 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/34 file after the snapshot 2024-08-18T13:41:40.771 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/35 file after the snapshot 2024-08-18T13:41:40.774 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/36 file after the snapshot 2024-08-18T13:41:40.777 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/37 file after the snapshot 2024-08-18T13:41:40.780 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/38 file after the snapshot 2024-08-18T13:41:40.783 INFO:tasks.workunit.client.0.smithi118.stdout: create 153/39 file after the snapshot 2024-08-18T13:41:40.786 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/.snap/snap-subdir-test 2024-08-18T13:41:40.864 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/30 file after the snapshot 2024-08-18T13:41:40.869 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/31 file after the snapshot 2024-08-18T13:41:40.874 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/32 file after the snapshot 2024-08-18T13:41:40.878 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/33 file after the snapshot 2024-08-18T13:41:40.881 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/34 file after the snapshot 2024-08-18T13:41:40.884 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/35 file after the snapshot 2024-08-18T13:41:40.887 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/36 file after the snapshot 2024-08-18T13:41:40.890 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/37 file after the snapshot 2024-08-18T13:41:40.893 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/38 file after the snapshot 2024-08-18T13:41:40.896 INFO:tasks.workunit.client.0.smithi118.stdout: create 154/39 file after the snapshot 2024-08-18T13:41:40.898 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/.snap/snap-subdir-test 2024-08-18T13:41:40.936 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/30 file after the snapshot 2024-08-18T13:41:40.940 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/31 file after the snapshot 2024-08-18T13:41:40.946 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/32 file after the snapshot 2024-08-18T13:41:40.951 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/33 file after the snapshot 2024-08-18T13:41:40.954 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/34 file after the snapshot 2024-08-18T13:41:40.958 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/35 file after the snapshot 2024-08-18T13:41:40.961 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/36 file after the snapshot 2024-08-18T13:41:40.964 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/37 file after the snapshot 2024-08-18T13:41:40.967 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/38 file after the snapshot 2024-08-18T13:41:40.970 INFO:tasks.workunit.client.0.smithi118.stdout: create 155/39 file after the snapshot 2024-08-18T13:41:40.972 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/.snap/snap-subdir-test 2024-08-18T13:41:41.005 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/30 file after the snapshot 2024-08-18T13:41:41.009 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/31 file after the snapshot 2024-08-18T13:41:41.013 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/32 file after the snapshot 2024-08-18T13:41:41.018 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/33 file after the snapshot 2024-08-18T13:41:41.021 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/34 file after the snapshot 2024-08-18T13:41:41.024 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/35 file after the snapshot 2024-08-18T13:41:41.027 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/36 file after the snapshot 2024-08-18T13:41:41.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/37 file after the snapshot 2024-08-18T13:41:41.034 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/38 file after the snapshot 2024-08-18T13:41:41.038 INFO:tasks.workunit.client.0.smithi118.stdout: create 156/39 file after the snapshot 2024-08-18T13:41:41.041 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/.snap/snap-subdir-test 2024-08-18T13:41:41.074 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/30 file after the snapshot 2024-08-18T13:41:41.078 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/31 file after the snapshot 2024-08-18T13:41:41.082 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/32 file after the snapshot 2024-08-18T13:41:41.087 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/33 file after the snapshot 2024-08-18T13:41:41.091 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/34 file after the snapshot 2024-08-18T13:41:41.094 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/35 file after the snapshot 2024-08-18T13:41:41.097 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/36 file after the snapshot 2024-08-18T13:41:41.100 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/37 file after the snapshot 2024-08-18T13:41:41.103 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/38 file after the snapshot 2024-08-18T13:41:41.107 INFO:tasks.workunit.client.0.smithi118.stdout: create 157/39 file after the snapshot 2024-08-18T13:41:41.110 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/.snap/snap-subdir-test 2024-08-18T13:41:41.143 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/30 file after the snapshot 2024-08-18T13:41:41.147 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/31 file after the snapshot 2024-08-18T13:41:41.152 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/32 file after the snapshot 2024-08-18T13:41:41.156 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/33 file after the snapshot 2024-08-18T13:41:41.160 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/34 file after the snapshot 2024-08-18T13:41:41.164 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/35 file after the snapshot 2024-08-18T13:41:41.167 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/36 file after the snapshot 2024-08-18T13:41:41.170 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/37 file after the snapshot 2024-08-18T13:41:41.173 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/38 file after the snapshot 2024-08-18T13:41:41.177 INFO:tasks.workunit.client.0.smithi118.stdout: create 158/39 file after the snapshot 2024-08-18T13:41:41.180 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/.snap/snap-subdir-test 2024-08-18T13:41:41.211 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/30 file after the snapshot 2024-08-18T13:41:41.214 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/31 file after the snapshot 2024-08-18T13:41:41.217 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/32 file after the snapshot 2024-08-18T13:41:41.221 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/33 file after the snapshot 2024-08-18T13:41:41.225 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/34 file after the snapshot 2024-08-18T13:41:41.229 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/35 file after the snapshot 2024-08-18T13:41:41.231 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/36 file after the snapshot 2024-08-18T13:41:41.234 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/37 file after the snapshot 2024-08-18T13:41:41.238 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/38 file after the snapshot 2024-08-18T13:41:41.240 INFO:tasks.workunit.client.0.smithi118.stdout: create 159/39 file after the snapshot 2024-08-18T13:41:41.243 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/.snap/snap-subdir-test 2024-08-18T13:41:41.272 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/30 file after the snapshot 2024-08-18T13:41:41.275 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/31 file after the snapshot 2024-08-18T13:41:41.281 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/32 file after the snapshot 2024-08-18T13:41:41.286 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/33 file after the snapshot 2024-08-18T13:41:41.289 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/34 file after the snapshot 2024-08-18T13:41:41.292 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/35 file after the snapshot 2024-08-18T13:41:41.295 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/36 file after the snapshot 2024-08-18T13:41:41.299 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/37 file after the snapshot 2024-08-18T13:41:41.302 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/38 file after the snapshot 2024-08-18T13:41:41.305 INFO:tasks.workunit.client.0.smithi118.stdout: create 160/39 file after the snapshot 2024-08-18T13:41:41.308 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/.snap/snap-subdir-test 2024-08-18T13:41:41.337 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/30 file after the snapshot 2024-08-18T13:41:41.340 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/31 file after the snapshot 2024-08-18T13:41:41.344 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/32 file after the snapshot 2024-08-18T13:41:41.349 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/33 file after the snapshot 2024-08-18T13:41:41.353 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/34 file after the snapshot 2024-08-18T13:41:41.356 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/35 file after the snapshot 2024-08-18T13:41:41.360 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/36 file after the snapshot 2024-08-18T13:41:41.363 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/37 file after the snapshot 2024-08-18T13:41:41.367 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/38 file after the snapshot 2024-08-18T13:41:41.370 INFO:tasks.workunit.client.0.smithi118.stdout: create 161/39 file after the snapshot 2024-08-18T13:41:41.373 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/.snap/snap-subdir-test 2024-08-18T13:41:41.414 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/30 file after the snapshot 2024-08-18T13:41:41.418 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/31 file after the snapshot 2024-08-18T13:41:41.422 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/32 file after the snapshot 2024-08-18T13:41:41.428 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/33 file after the snapshot 2024-08-18T13:41:41.432 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/34 file after the snapshot 2024-08-18T13:41:41.435 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/35 file after the snapshot 2024-08-18T13:41:41.438 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/36 file after the snapshot 2024-08-18T13:41:41.442 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/37 file after the snapshot 2024-08-18T13:41:41.445 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/38 file after the snapshot 2024-08-18T13:41:41.448 INFO:tasks.workunit.client.0.smithi118.stdout: create 162/39 file after the snapshot 2024-08-18T13:41:41.450 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/.snap/snap-subdir-test 2024-08-18T13:41:41.483 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/30 file after the snapshot 2024-08-18T13:41:41.487 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/31 file after the snapshot 2024-08-18T13:41:41.491 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/32 file after the snapshot 2024-08-18T13:41:41.495 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/33 file after the snapshot 2024-08-18T13:41:41.499 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/34 file after the snapshot 2024-08-18T13:41:41.503 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/35 file after the snapshot 2024-08-18T13:41:41.507 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/36 file after the snapshot 2024-08-18T13:41:41.510 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/37 file after the snapshot 2024-08-18T13:41:41.512 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/38 file after the snapshot 2024-08-18T13:41:41.518 INFO:tasks.workunit.client.0.smithi118.stdout: create 163/39 file after the snapshot 2024-08-18T13:41:41.520 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/.snap/snap-subdir-test 2024-08-18T13:41:41.551 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/30 file after the snapshot 2024-08-18T13:41:41.555 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/31 file after the snapshot 2024-08-18T13:41:41.559 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/32 file after the snapshot 2024-08-18T13:41:41.564 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/33 file after the snapshot 2024-08-18T13:41:41.567 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/34 file after the snapshot 2024-08-18T13:41:41.571 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/35 file after the snapshot 2024-08-18T13:41:41.574 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/36 file after the snapshot 2024-08-18T13:41:41.578 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/37 file after the snapshot 2024-08-18T13:41:41.580 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/38 file after the snapshot 2024-08-18T13:41:41.584 INFO:tasks.workunit.client.0.smithi118.stdout: create 164/39 file after the snapshot 2024-08-18T13:41:41.586 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/.snap/snap-subdir-test 2024-08-18T13:41:41.615 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/30 file after the snapshot 2024-08-18T13:41:41.620 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/31 file after the snapshot 2024-08-18T13:41:41.624 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/32 file after the snapshot 2024-08-18T13:41:41.629 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/33 file after the snapshot 2024-08-18T13:41:41.632 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/34 file after the snapshot 2024-08-18T13:41:41.636 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/35 file after the snapshot 2024-08-18T13:41:41.640 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/36 file after the snapshot 2024-08-18T13:41:41.643 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/37 file after the snapshot 2024-08-18T13:41:41.645 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/38 file after the snapshot 2024-08-18T13:41:41.648 INFO:tasks.workunit.client.0.smithi118.stdout: create 165/39 file after the snapshot 2024-08-18T13:41:41.651 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/.snap/snap-subdir-test 2024-08-18T13:41:41.710 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/30 file after the snapshot 2024-08-18T13:41:41.829 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/31 file after the snapshot 2024-08-18T13:41:41.833 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/32 file after the snapshot 2024-08-18T13:41:41.836 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/33 file after the snapshot 2024-08-18T13:41:41.840 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/34 file after the snapshot 2024-08-18T13:41:41.843 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/35 file after the snapshot 2024-08-18T13:41:41.845 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/36 file after the snapshot 2024-08-18T13:41:41.848 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/37 file after the snapshot 2024-08-18T13:41:41.850 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/38 file after the snapshot 2024-08-18T13:41:41.853 INFO:tasks.workunit.client.0.smithi118.stdout: create 166/39 file after the snapshot 2024-08-18T13:41:41.856 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/.snap/snap-subdir-test 2024-08-18T13:41:41.889 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:41:41.892 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:41:41.934 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/30 file after the snapshot 2024-08-18T13:41:41.935 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/31 file after the snapshot 2024-08-18T13:41:41.940 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/32 file after the snapshot 2024-08-18T13:41:41.945 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/33 file after the snapshot 2024-08-18T13:41:41.949 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/34 file after the snapshot 2024-08-18T13:41:41.953 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/35 file after the snapshot 2024-08-18T13:41:41.955 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/36 file after the snapshot 2024-08-18T13:41:41.959 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/37 file after the snapshot 2024-08-18T13:41:41.962 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/38 file after the snapshot 2024-08-18T13:41:41.966 INFO:tasks.workunit.client.0.smithi118.stdout: create 167/39 file after the snapshot 2024-08-18T13:41:41.968 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/.snap/snap-subdir-test 2024-08-18T13:41:41.997 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/30 file after the snapshot 2024-08-18T13:41:42.000 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/31 file after the snapshot 2024-08-18T13:41:42.004 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/32 file after the snapshot 2024-08-18T13:41:42.007 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/33 file after the snapshot 2024-08-18T13:41:42.012 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/34 file after the snapshot 2024-08-18T13:41:42.015 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/35 file after the snapshot 2024-08-18T13:41:42.018 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/36 file after the snapshot 2024-08-18T13:41:42.021 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/37 file after the snapshot 2024-08-18T13:41:42.023 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/38 file after the snapshot 2024-08-18T13:41:42.027 INFO:tasks.workunit.client.0.smithi118.stdout: create 168/39 file after the snapshot 2024-08-18T13:41:42.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/.snap/snap-subdir-test 2024-08-18T13:41:42.056 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/30 file after the snapshot 2024-08-18T13:41:42.060 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/31 file after the snapshot 2024-08-18T13:41:42.064 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/32 file after the snapshot 2024-08-18T13:41:42.069 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/33 file after the snapshot 2024-08-18T13:41:42.073 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/34 file after the snapshot 2024-08-18T13:41:42.076 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/35 file after the snapshot 2024-08-18T13:41:42.079 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/36 file after the snapshot 2024-08-18T13:41:42.082 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/37 file after the snapshot 2024-08-18T13:41:42.085 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/38 file after the snapshot 2024-08-18T13:41:42.088 INFO:tasks.workunit.client.0.smithi118.stdout: create 169/39 file after the snapshot 2024-08-18T13:41:42.092 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/.snap/snap-subdir-test 2024-08-18T13:41:42.121 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/30 file after the snapshot 2024-08-18T13:41:42.125 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/31 file after the snapshot 2024-08-18T13:41:42.130 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/32 file after the snapshot 2024-08-18T13:41:42.135 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/33 file after the snapshot 2024-08-18T13:41:42.138 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/34 file after the snapshot 2024-08-18T13:41:42.141 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/35 file after the snapshot 2024-08-18T13:41:42.144 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/36 file after the snapshot 2024-08-18T13:41:42.147 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/37 file after the snapshot 2024-08-18T13:41:42.150 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/38 file after the snapshot 2024-08-18T13:41:42.153 INFO:tasks.workunit.client.0.smithi118.stdout: create 170/39 file after the snapshot 2024-08-18T13:41:42.156 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/.snap/snap-subdir-test 2024-08-18T13:41:42.187 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/30 file after the snapshot 2024-08-18T13:41:42.191 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/31 file after the snapshot 2024-08-18T13:41:42.194 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/32 file after the snapshot 2024-08-18T13:41:42.200 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/33 file after the snapshot 2024-08-18T13:41:42.204 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/34 file after the snapshot 2024-08-18T13:41:42.207 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/35 file after the snapshot 2024-08-18T13:41:42.211 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/36 file after the snapshot 2024-08-18T13:41:42.214 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/37 file after the snapshot 2024-08-18T13:41:42.217 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/38 file after the snapshot 2024-08-18T13:41:42.219 INFO:tasks.workunit.client.0.smithi118.stdout: create 171/39 file after the snapshot 2024-08-18T13:41:42.222 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/.snap/snap-subdir-test 2024-08-18T13:41:42.251 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/30 file after the snapshot 2024-08-18T13:41:42.254 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/31 file after the snapshot 2024-08-18T13:41:42.258 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/32 file after the snapshot 2024-08-18T13:41:42.263 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/33 file after the snapshot 2024-08-18T13:41:42.267 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/34 file after the snapshot 2024-08-18T13:41:42.271 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/35 file after the snapshot 2024-08-18T13:41:42.274 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/36 file after the snapshot 2024-08-18T13:41:42.277 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/37 file after the snapshot 2024-08-18T13:41:42.280 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/38 file after the snapshot 2024-08-18T13:41:42.282 INFO:tasks.workunit.client.0.smithi118.stdout: create 172/39 file after the snapshot 2024-08-18T13:41:42.286 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/.snap/snap-subdir-test 2024-08-18T13:41:42.313 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/30 file after the snapshot 2024-08-18T13:41:42.318 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/31 file after the snapshot 2024-08-18T13:41:42.322 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/32 file after the snapshot 2024-08-18T13:41:42.327 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/33 file after the snapshot 2024-08-18T13:41:42.330 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/34 file after the snapshot 2024-08-18T13:41:42.334 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/35 file after the snapshot 2024-08-18T13:41:42.337 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/36 file after the snapshot 2024-08-18T13:41:42.342 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/37 file after the snapshot 2024-08-18T13:41:42.343 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/38 file after the snapshot 2024-08-18T13:41:42.345 INFO:tasks.workunit.client.0.smithi118.stdout: create 173/39 file after the snapshot 2024-08-18T13:41:42.348 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/.snap/snap-subdir-test 2024-08-18T13:41:42.375 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/30 file after the snapshot 2024-08-18T13:41:42.380 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/31 file after the snapshot 2024-08-18T13:41:42.384 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/32 file after the snapshot 2024-08-18T13:41:42.388 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/33 file after the snapshot 2024-08-18T13:41:42.392 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/34 file after the snapshot 2024-08-18T13:41:42.395 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/35 file after the snapshot 2024-08-18T13:41:42.398 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/36 file after the snapshot 2024-08-18T13:41:42.401 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/37 file after the snapshot 2024-08-18T13:41:42.404 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/38 file after the snapshot 2024-08-18T13:41:42.406 INFO:tasks.workunit.client.0.smithi118.stdout: create 174/39 file after the snapshot 2024-08-18T13:41:42.409 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/.snap/snap-subdir-test 2024-08-18T13:41:42.446 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/30 file after the snapshot 2024-08-18T13:41:42.450 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/31 file after the snapshot 2024-08-18T13:41:42.454 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/32 file after the snapshot 2024-08-18T13:41:42.458 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/33 file after the snapshot 2024-08-18T13:41:42.462 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/34 file after the snapshot 2024-08-18T13:41:42.465 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/35 file after the snapshot 2024-08-18T13:41:42.469 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/36 file after the snapshot 2024-08-18T13:41:42.471 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/37 file after the snapshot 2024-08-18T13:41:42.474 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/38 file after the snapshot 2024-08-18T13:41:42.477 INFO:tasks.workunit.client.0.smithi118.stdout: create 175/39 file after the snapshot 2024-08-18T13:41:42.479 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/.snap/snap-subdir-test 2024-08-18T13:41:42.507 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/30 file after the snapshot 2024-08-18T13:41:42.510 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/31 file after the snapshot 2024-08-18T13:41:42.514 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/32 file after the snapshot 2024-08-18T13:41:42.519 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/33 file after the snapshot 2024-08-18T13:41:42.524 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/34 file after the snapshot 2024-08-18T13:41:42.527 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/35 file after the snapshot 2024-08-18T13:41:42.530 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/36 file after the snapshot 2024-08-18T13:41:42.533 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/37 file after the snapshot 2024-08-18T13:41:42.536 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/38 file after the snapshot 2024-08-18T13:41:42.539 INFO:tasks.workunit.client.0.smithi118.stdout: create 176/39 file after the snapshot 2024-08-18T13:41:42.543 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/.snap/snap-subdir-test 2024-08-18T13:41:42.572 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/30 file after the snapshot 2024-08-18T13:41:42.576 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/31 file after the snapshot 2024-08-18T13:41:42.580 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/32 file after the snapshot 2024-08-18T13:41:42.586 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/33 file after the snapshot 2024-08-18T13:41:42.589 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/34 file after the snapshot 2024-08-18T13:41:42.593 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/35 file after the snapshot 2024-08-18T13:41:42.596 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/36 file after the snapshot 2024-08-18T13:41:42.599 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/37 file after the snapshot 2024-08-18T13:41:42.602 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/38 file after the snapshot 2024-08-18T13:41:42.605 INFO:tasks.workunit.client.0.smithi118.stdout: create 177/39 file after the snapshot 2024-08-18T13:41:42.609 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/.snap/snap-subdir-test 2024-08-18T13:41:42.636 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/30 file after the snapshot 2024-08-18T13:41:42.640 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/31 file after the snapshot 2024-08-18T13:41:42.645 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/32 file after the snapshot 2024-08-18T13:41:42.649 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/33 file after the snapshot 2024-08-18T13:41:42.653 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/34 file after the snapshot 2024-08-18T13:41:42.657 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/35 file after the snapshot 2024-08-18T13:41:42.660 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/36 file after the snapshot 2024-08-18T13:41:42.662 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/37 file after the snapshot 2024-08-18T13:41:42.666 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/38 file after the snapshot 2024-08-18T13:41:42.669 INFO:tasks.workunit.client.0.smithi118.stdout: create 178/39 file after the snapshot 2024-08-18T13:41:42.672 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/.snap/snap-subdir-test 2024-08-18T13:41:42.769 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/30 file after the snapshot 2024-08-18T13:41:42.773 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/31 file after the snapshot 2024-08-18T13:41:42.779 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/32 file after the snapshot 2024-08-18T13:41:42.783 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/33 file after the snapshot 2024-08-18T13:41:42.786 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/34 file after the snapshot 2024-08-18T13:41:42.789 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/35 file after the snapshot 2024-08-18T13:41:42.793 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/36 file after the snapshot 2024-08-18T13:41:42.797 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/37 file after the snapshot 2024-08-18T13:41:42.800 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/38 file after the snapshot 2024-08-18T13:41:42.803 INFO:tasks.workunit.client.0.smithi118.stdout: create 179/39 file after the snapshot 2024-08-18T13:41:42.806 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/.snap/snap-subdir-test 2024-08-18T13:41:42.837 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/30 file after the snapshot 2024-08-18T13:41:42.842 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/31 file after the snapshot 2024-08-18T13:41:42.846 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/32 file after the snapshot 2024-08-18T13:41:42.851 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/33 file after the snapshot 2024-08-18T13:41:42.855 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/34 file after the snapshot 2024-08-18T13:41:42.858 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/35 file after the snapshot 2024-08-18T13:41:42.862 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/36 file after the snapshot 2024-08-18T13:41:42.865 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/37 file after the snapshot 2024-08-18T13:41:42.868 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/38 file after the snapshot 2024-08-18T13:41:42.871 INFO:tasks.workunit.client.0.smithi118.stdout: create 180/39 file after the snapshot 2024-08-18T13:41:42.874 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/.snap/snap-subdir-test 2024-08-18T13:41:42.903 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/30 file after the snapshot 2024-08-18T13:41:42.907 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/31 file after the snapshot 2024-08-18T13:41:42.911 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/32 file after the snapshot 2024-08-18T13:41:42.915 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/33 file after the snapshot 2024-08-18T13:41:42.919 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/34 file after the snapshot 2024-08-18T13:41:42.922 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/35 file after the snapshot 2024-08-18T13:41:42.925 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/36 file after the snapshot 2024-08-18T13:41:42.928 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/37 file after the snapshot 2024-08-18T13:41:42.931 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/38 file after the snapshot 2024-08-18T13:41:42.934 INFO:tasks.workunit.client.0.smithi118.stdout: create 181/39 file after the snapshot 2024-08-18T13:41:42.937 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/.snap/snap-subdir-test 2024-08-18T13:41:42.965 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/30 file after the snapshot 2024-08-18T13:41:42.969 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/31 file after the snapshot 2024-08-18T13:41:42.975 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/32 file after the snapshot 2024-08-18T13:41:42.979 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/33 file after the snapshot 2024-08-18T13:41:42.983 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/34 file after the snapshot 2024-08-18T13:41:42.986 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/35 file after the snapshot 2024-08-18T13:41:42.988 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/36 file after the snapshot 2024-08-18T13:41:42.991 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/37 file after the snapshot 2024-08-18T13:41:42.994 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/38 file after the snapshot 2024-08-18T13:41:42.998 INFO:tasks.workunit.client.0.smithi118.stdout: create 182/39 file after the snapshot 2024-08-18T13:41:43.001 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/.snap/snap-subdir-test 2024-08-18T13:41:43.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/30 file after the snapshot 2024-08-18T13:41:43.033 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/31 file after the snapshot 2024-08-18T13:41:43.038 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/32 file after the snapshot 2024-08-18T13:41:43.042 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/33 file after the snapshot 2024-08-18T13:41:43.046 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/34 file after the snapshot 2024-08-18T13:41:43.050 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/35 file after the snapshot 2024-08-18T13:41:43.053 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/36 file after the snapshot 2024-08-18T13:41:43.055 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/37 file after the snapshot 2024-08-18T13:41:43.059 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/38 file after the snapshot 2024-08-18T13:41:43.062 INFO:tasks.workunit.client.0.smithi118.stdout: create 183/39 file after the snapshot 2024-08-18T13:41:43.065 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/.snap/snap-subdir-test 2024-08-18T13:41:43.093 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/30 file after the snapshot 2024-08-18T13:41:43.097 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/31 file after the snapshot 2024-08-18T13:41:43.101 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/32 file after the snapshot 2024-08-18T13:41:43.105 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/33 file after the snapshot 2024-08-18T13:41:43.108 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/34 file after the snapshot 2024-08-18T13:41:43.111 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/35 file after the snapshot 2024-08-18T13:41:43.115 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/36 file after the snapshot 2024-08-18T13:41:43.118 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/37 file after the snapshot 2024-08-18T13:41:43.121 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/38 file after the snapshot 2024-08-18T13:41:43.124 INFO:tasks.workunit.client.0.smithi118.stdout: create 184/39 file after the snapshot 2024-08-18T13:41:43.127 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/.snap/snap-subdir-test 2024-08-18T13:41:43.156 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/30 file after the snapshot 2024-08-18T13:41:43.162 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/31 file after the snapshot 2024-08-18T13:41:43.165 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/32 file after the snapshot 2024-08-18T13:41:43.172 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/33 file after the snapshot 2024-08-18T13:41:43.176 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/34 file after the snapshot 2024-08-18T13:41:43.179 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/35 file after the snapshot 2024-08-18T13:41:43.182 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/36 file after the snapshot 2024-08-18T13:41:43.185 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/37 file after the snapshot 2024-08-18T13:41:43.188 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/38 file after the snapshot 2024-08-18T13:41:43.191 INFO:tasks.workunit.client.0.smithi118.stdout: create 185/39 file after the snapshot 2024-08-18T13:41:43.194 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/.snap/snap-subdir-test 2024-08-18T13:41:43.221 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/30 file after the snapshot 2024-08-18T13:41:43.224 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/31 file after the snapshot 2024-08-18T13:41:43.228 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/32 file after the snapshot 2024-08-18T13:41:43.233 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/33 file after the snapshot 2024-08-18T13:41:43.237 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/34 file after the snapshot 2024-08-18T13:41:43.240 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/35 file after the snapshot 2024-08-18T13:41:43.244 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/36 file after the snapshot 2024-08-18T13:41:43.247 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/37 file after the snapshot 2024-08-18T13:41:43.250 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/38 file after the snapshot 2024-08-18T13:41:43.252 INFO:tasks.workunit.client.0.smithi118.stdout: create 186/39 file after the snapshot 2024-08-18T13:41:43.255 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/.snap/snap-subdir-test 2024-08-18T13:41:43.290 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/30 file after the snapshot 2024-08-18T13:41:43.295 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/31 file after the snapshot 2024-08-18T13:41:43.300 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/32 file after the snapshot 2024-08-18T13:41:43.305 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/33 file after the snapshot 2024-08-18T13:41:43.308 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/34 file after the snapshot 2024-08-18T13:41:43.311 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/35 file after the snapshot 2024-08-18T13:41:43.315 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/36 file after the snapshot 2024-08-18T13:41:43.317 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/37 file after the snapshot 2024-08-18T13:41:43.320 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/38 file after the snapshot 2024-08-18T13:41:43.323 INFO:tasks.workunit.client.0.smithi118.stdout: create 187/39 file after the snapshot 2024-08-18T13:41:43.326 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/.snap/snap-subdir-test 2024-08-18T13:41:43.360 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/30 file after the snapshot 2024-08-18T13:41:43.364 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/31 file after the snapshot 2024-08-18T13:41:43.368 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/32 file after the snapshot 2024-08-18T13:41:43.374 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/33 file after the snapshot 2024-08-18T13:41:43.378 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/34 file after the snapshot 2024-08-18T13:41:43.382 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/35 file after the snapshot 2024-08-18T13:41:43.384 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/36 file after the snapshot 2024-08-18T13:41:43.387 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/37 file after the snapshot 2024-08-18T13:41:43.390 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/38 file after the snapshot 2024-08-18T13:41:43.393 INFO:tasks.workunit.client.0.smithi118.stdout: create 188/39 file after the snapshot 2024-08-18T13:41:43.396 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/.snap/snap-subdir-test 2024-08-18T13:41:43.424 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/30 file after the snapshot 2024-08-18T13:41:43.428 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/31 file after the snapshot 2024-08-18T13:41:43.432 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/32 file after the snapshot 2024-08-18T13:41:43.436 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/33 file after the snapshot 2024-08-18T13:41:43.440 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/34 file after the snapshot 2024-08-18T13:41:43.444 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/35 file after the snapshot 2024-08-18T13:41:43.447 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/36 file after the snapshot 2024-08-18T13:41:43.450 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/37 file after the snapshot 2024-08-18T13:41:43.453 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/38 file after the snapshot 2024-08-18T13:41:43.456 INFO:tasks.workunit.client.0.smithi118.stdout: create 189/39 file after the snapshot 2024-08-18T13:41:43.459 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/.snap/snap-subdir-test 2024-08-18T13:41:43.492 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/30 file after the snapshot 2024-08-18T13:41:43.496 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/31 file after the snapshot 2024-08-18T13:41:43.502 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/32 file after the snapshot 2024-08-18T13:41:43.507 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/33 file after the snapshot 2024-08-18T13:41:43.510 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/34 file after the snapshot 2024-08-18T13:41:43.513 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/35 file after the snapshot 2024-08-18T13:41:43.516 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/36 file after the snapshot 2024-08-18T13:41:43.522 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/37 file after the snapshot 2024-08-18T13:41:43.524 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/38 file after the snapshot 2024-08-18T13:41:43.527 INFO:tasks.workunit.client.0.smithi118.stdout: create 190/39 file after the snapshot 2024-08-18T13:41:43.530 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/.snap/snap-subdir-test 2024-08-18T13:41:43.559 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/30 file after the snapshot 2024-08-18T13:41:43.563 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/31 file after the snapshot 2024-08-18T13:41:43.570 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/32 file after the snapshot 2024-08-18T13:41:43.571 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/33 file after the snapshot 2024-08-18T13:41:43.574 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/34 file after the snapshot 2024-08-18T13:41:43.578 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/35 file after the snapshot 2024-08-18T13:41:43.581 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/36 file after the snapshot 2024-08-18T13:41:43.584 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/37 file after the snapshot 2024-08-18T13:41:43.587 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/38 file after the snapshot 2024-08-18T13:41:43.589 INFO:tasks.workunit.client.0.smithi118.stdout: create 191/39 file after the snapshot 2024-08-18T13:41:43.592 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/.snap/snap-subdir-test 2024-08-18T13:41:43.620 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/30 file after the snapshot 2024-08-18T13:41:43.623 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/31 file after the snapshot 2024-08-18T13:41:43.628 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/32 file after the snapshot 2024-08-18T13:41:43.633 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/33 file after the snapshot 2024-08-18T13:41:43.637 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/34 file after the snapshot 2024-08-18T13:41:43.640 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/35 file after the snapshot 2024-08-18T13:41:43.643 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/36 file after the snapshot 2024-08-18T13:41:43.646 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/37 file after the snapshot 2024-08-18T13:41:43.649 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/38 file after the snapshot 2024-08-18T13:41:43.652 INFO:tasks.workunit.client.0.smithi118.stdout: create 192/39 file after the snapshot 2024-08-18T13:41:43.655 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/.snap/snap-subdir-test 2024-08-18T13:41:43.681 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/30 file after the snapshot 2024-08-18T13:41:43.685 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/31 file after the snapshot 2024-08-18T13:41:43.700 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/32 file after the snapshot 2024-08-18T13:41:43.757 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/33 file after the snapshot 2024-08-18T13:41:43.761 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/34 file after the snapshot 2024-08-18T13:41:43.765 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/35 file after the snapshot 2024-08-18T13:41:43.769 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/36 file after the snapshot 2024-08-18T13:41:43.773 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/37 file after the snapshot 2024-08-18T13:41:43.778 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/38 file after the snapshot 2024-08-18T13:41:43.779 INFO:tasks.workunit.client.0.smithi118.stdout: create 193/39 file after the snapshot 2024-08-18T13:41:43.783 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/.snap/snap-subdir-test 2024-08-18T13:41:43.841 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/30 file after the snapshot 2024-08-18T13:41:43.845 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/31 file after the snapshot 2024-08-18T13:41:43.850 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/32 file after the snapshot 2024-08-18T13:41:43.854 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/33 file after the snapshot 2024-08-18T13:41:43.858 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/34 file after the snapshot 2024-08-18T13:41:43.860 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/35 file after the snapshot 2024-08-18T13:41:43.864 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/36 file after the snapshot 2024-08-18T13:41:43.866 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/37 file after the snapshot 2024-08-18T13:41:43.869 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/38 file after the snapshot 2024-08-18T13:41:43.872 INFO:tasks.workunit.client.0.smithi118.stdout: create 194/39 file after the snapshot 2024-08-18T13:41:43.876 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/.snap/snap-subdir-test 2024-08-18T13:41:43.905 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/30 file after the snapshot 2024-08-18T13:41:43.909 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/31 file after the snapshot 2024-08-18T13:41:43.914 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/32 file after the snapshot 2024-08-18T13:41:43.919 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/33 file after the snapshot 2024-08-18T13:41:43.921 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/34 file after the snapshot 2024-08-18T13:41:43.925 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/35 file after the snapshot 2024-08-18T13:41:43.927 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/36 file after the snapshot 2024-08-18T13:41:43.930 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/37 file after the snapshot 2024-08-18T13:41:43.932 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/38 file after the snapshot 2024-08-18T13:41:43.935 INFO:tasks.workunit.client.0.smithi118.stdout: create 195/39 file after the snapshot 2024-08-18T13:41:43.938 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/.snap/snap-subdir-test 2024-08-18T13:41:43.963 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/30 file after the snapshot 2024-08-18T13:41:43.967 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/31 file after the snapshot 2024-08-18T13:41:43.971 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/32 file after the snapshot 2024-08-18T13:41:43.976 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/33 file after the snapshot 2024-08-18T13:41:43.979 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/34 file after the snapshot 2024-08-18T13:41:43.983 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/35 file after the snapshot 2024-08-18T13:41:43.986 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/36 file after the snapshot 2024-08-18T13:41:43.989 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/37 file after the snapshot 2024-08-18T13:41:43.991 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/38 file after the snapshot 2024-08-18T13:41:43.993 INFO:tasks.workunit.client.0.smithi118.stdout: create 196/39 file after the snapshot 2024-08-18T13:41:43.996 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/.snap/snap-subdir-test 2024-08-18T13:41:44.025 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/30 file after the snapshot 2024-08-18T13:41:44.028 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/31 file after the snapshot 2024-08-18T13:41:44.034 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/32 file after the snapshot 2024-08-18T13:41:44.036 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/33 file after the snapshot 2024-08-18T13:41:44.040 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/34 file after the snapshot 2024-08-18T13:41:44.043 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/35 file after the snapshot 2024-08-18T13:41:44.046 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/36 file after the snapshot 2024-08-18T13:41:44.049 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/37 file after the snapshot 2024-08-18T13:41:44.052 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/38 file after the snapshot 2024-08-18T13:41:44.055 INFO:tasks.workunit.client.0.smithi118.stdout: create 197/39 file after the snapshot 2024-08-18T13:41:44.057 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/.snap/snap-subdir-test 2024-08-18T13:41:44.083 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/30 file after the snapshot 2024-08-18T13:41:44.087 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/31 file after the snapshot 2024-08-18T13:41:44.090 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/32 file after the snapshot 2024-08-18T13:41:44.094 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/33 file after the snapshot 2024-08-18T13:41:44.098 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/34 file after the snapshot 2024-08-18T13:41:44.102 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/35 file after the snapshot 2024-08-18T13:41:44.105 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/36 file after the snapshot 2024-08-18T13:41:44.109 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/37 file after the snapshot 2024-08-18T13:41:44.111 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/38 file after the snapshot 2024-08-18T13:41:44.115 INFO:tasks.workunit.client.0.smithi118.stdout: create 198/39 file after the snapshot 2024-08-18T13:41:44.118 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/.snap/snap-subdir-test 2024-08-18T13:41:44.146 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/30 file after the snapshot 2024-08-18T13:41:44.151 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/31 file after the snapshot 2024-08-18T13:41:44.155 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/32 file after the snapshot 2024-08-18T13:41:44.159 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/33 file after the snapshot 2024-08-18T13:41:44.163 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/34 file after the snapshot 2024-08-18T13:41:44.167 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/35 file after the snapshot 2024-08-18T13:41:44.170 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/36 file after the snapshot 2024-08-18T13:41:44.172 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/37 file after the snapshot 2024-08-18T13:41:44.175 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/38 file after the snapshot 2024-08-18T13:41:44.178 INFO:tasks.workunit.client.0.smithi118.stdout: create 199/39 file after the snapshot 2024-08-18T13:41:44.181 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/.snap/snap-subdir-test 2024-08-18T13:41:44.206 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/30 file after the snapshot 2024-08-18T13:41:44.209 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/31 file after the snapshot 2024-08-18T13:41:44.213 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/32 file after the snapshot 2024-08-18T13:41:44.217 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/33 file after the snapshot 2024-08-18T13:41:44.221 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/34 file after the snapshot 2024-08-18T13:41:44.224 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/35 file after the snapshot 2024-08-18T13:41:44.227 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/36 file after the snapshot 2024-08-18T13:41:44.230 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/37 file after the snapshot 2024-08-18T13:41:44.233 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/38 file after the snapshot 2024-08-18T13:41:44.235 INFO:tasks.workunit.client.0.smithi118.stdout: create 200/39 file after the snapshot 2024-08-18T13:41:44.238 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/.snap/snap-subdir-test 2024-08-18T13:41:44.264 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/30 file after the snapshot 2024-08-18T13:41:44.268 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/31 file after the snapshot 2024-08-18T13:41:44.272 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/32 file after the snapshot 2024-08-18T13:41:44.276 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/33 file after the snapshot 2024-08-18T13:41:44.280 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/34 file after the snapshot 2024-08-18T13:41:44.284 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/35 file after the snapshot 2024-08-18T13:41:44.287 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/36 file after the snapshot 2024-08-18T13:41:44.290 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/37 file after the snapshot 2024-08-18T13:41:44.293 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/38 file after the snapshot 2024-08-18T13:41:44.296 INFO:tasks.workunit.client.0.smithi118.stdout: create 201/39 file after the snapshot 2024-08-18T13:41:44.300 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/.snap/snap-subdir-test 2024-08-18T13:41:44.329 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/30 file after the snapshot 2024-08-18T13:41:44.333 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/31 file after the snapshot 2024-08-18T13:41:44.337 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/32 file after the snapshot 2024-08-18T13:41:44.342 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/33 file after the snapshot 2024-08-18T13:41:44.346 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/34 file after the snapshot 2024-08-18T13:41:44.349 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/35 file after the snapshot 2024-08-18T13:41:44.353 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/36 file after the snapshot 2024-08-18T13:41:44.355 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/37 file after the snapshot 2024-08-18T13:41:44.358 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/38 file after the snapshot 2024-08-18T13:41:44.361 INFO:tasks.workunit.client.0.smithi118.stdout: create 202/39 file after the snapshot 2024-08-18T13:41:44.364 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/.snap/snap-subdir-test 2024-08-18T13:41:44.391 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/30 file after the snapshot 2024-08-18T13:41:44.394 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/31 file after the snapshot 2024-08-18T13:41:44.397 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/32 file after the snapshot 2024-08-18T13:41:44.402 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/33 file after the snapshot 2024-08-18T13:41:44.405 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/34 file after the snapshot 2024-08-18T13:41:44.408 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/35 file after the snapshot 2024-08-18T13:41:44.412 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/36 file after the snapshot 2024-08-18T13:41:44.415 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/37 file after the snapshot 2024-08-18T13:41:44.418 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/38 file after the snapshot 2024-08-18T13:41:44.421 INFO:tasks.workunit.client.0.smithi118.stdout: create 203/39 file after the snapshot 2024-08-18T13:41:44.423 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/.snap/snap-subdir-test 2024-08-18T13:41:44.453 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/30 file after the snapshot 2024-08-18T13:41:44.456 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/31 file after the snapshot 2024-08-18T13:41:44.461 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/32 file after the snapshot 2024-08-18T13:41:44.464 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/33 file after the snapshot 2024-08-18T13:41:44.468 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/34 file after the snapshot 2024-08-18T13:41:44.471 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/35 file after the snapshot 2024-08-18T13:41:44.474 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/36 file after the snapshot 2024-08-18T13:41:44.477 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/37 file after the snapshot 2024-08-18T13:41:44.480 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/38 file after the snapshot 2024-08-18T13:41:44.483 INFO:tasks.workunit.client.0.smithi118.stdout: create 204/39 file after the snapshot 2024-08-18T13:41:44.486 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/.snap/snap-subdir-test 2024-08-18T13:41:44.516 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/30 file after the snapshot 2024-08-18T13:41:44.520 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/31 file after the snapshot 2024-08-18T13:41:44.524 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/32 file after the snapshot 2024-08-18T13:41:44.529 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/33 file after the snapshot 2024-08-18T13:41:44.531 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/34 file after the snapshot 2024-08-18T13:41:44.535 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/35 file after the snapshot 2024-08-18T13:41:44.538 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/36 file after the snapshot 2024-08-18T13:41:44.541 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/37 file after the snapshot 2024-08-18T13:41:44.544 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/38 file after the snapshot 2024-08-18T13:41:44.547 INFO:tasks.workunit.client.0.smithi118.stdout: create 205/39 file after the snapshot 2024-08-18T13:41:44.551 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/.snap/snap-subdir-test 2024-08-18T13:41:44.579 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/30 file after the snapshot 2024-08-18T13:41:44.582 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/31 file after the snapshot 2024-08-18T13:41:44.587 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/32 file after the snapshot 2024-08-18T13:41:44.591 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/33 file after the snapshot 2024-08-18T13:41:44.595 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/34 file after the snapshot 2024-08-18T13:41:44.599 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/35 file after the snapshot 2024-08-18T13:41:44.602 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/36 file after the snapshot 2024-08-18T13:41:44.605 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/37 file after the snapshot 2024-08-18T13:41:44.608 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/38 file after the snapshot 2024-08-18T13:41:44.612 INFO:tasks.workunit.client.0.smithi118.stdout: create 206/39 file after the snapshot 2024-08-18T13:41:44.614 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/.snap/snap-subdir-test 2024-08-18T13:41:44.641 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/30 file after the snapshot 2024-08-18T13:41:44.644 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/31 file after the snapshot 2024-08-18T13:41:44.649 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/32 file after the snapshot 2024-08-18T13:41:44.653 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/33 file after the snapshot 2024-08-18T13:41:44.657 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/34 file after the snapshot 2024-08-18T13:41:44.661 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/35 file after the snapshot 2024-08-18T13:41:44.664 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/36 file after the snapshot 2024-08-18T13:41:44.666 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/37 file after the snapshot 2024-08-18T13:41:44.670 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/38 file after the snapshot 2024-08-18T13:41:44.672 INFO:tasks.workunit.client.0.smithi118.stdout: create 207/39 file after the snapshot 2024-08-18T13:41:44.675 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/.snap/snap-subdir-test 2024-08-18T13:41:44.720 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/30 file after the snapshot 2024-08-18T13:41:44.783 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/31 file after the snapshot 2024-08-18T13:41:44.787 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/32 file after the snapshot 2024-08-18T13:41:44.791 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/33 file after the snapshot 2024-08-18T13:41:44.795 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/34 file after the snapshot 2024-08-18T13:41:44.798 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/35 file after the snapshot 2024-08-18T13:41:44.802 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/36 file after the snapshot 2024-08-18T13:41:44.804 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/37 file after the snapshot 2024-08-18T13:41:44.807 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/38 file after the snapshot 2024-08-18T13:41:44.810 INFO:tasks.workunit.client.0.smithi118.stdout: create 208/39 file after the snapshot 2024-08-18T13:41:44.814 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/.snap/snap-subdir-test 2024-08-18T13:41:44.873 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/30 file after the snapshot 2024-08-18T13:41:44.877 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/31 file after the snapshot 2024-08-18T13:41:44.881 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/32 file after the snapshot 2024-08-18T13:41:44.885 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/33 file after the snapshot 2024-08-18T13:41:44.889 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/34 file after the snapshot 2024-08-18T13:41:44.893 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/35 file after the snapshot 2024-08-18T13:41:44.896 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/36 file after the snapshot 2024-08-18T13:41:44.899 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/37 file after the snapshot 2024-08-18T13:41:44.902 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/38 file after the snapshot 2024-08-18T13:41:44.904 INFO:tasks.workunit.client.0.smithi118.stdout: create 209/39 file after the snapshot 2024-08-18T13:41:44.907 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/.snap/snap-subdir-test 2024-08-18T13:41:44.940 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/30 file after the snapshot 2024-08-18T13:41:44.944 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/31 file after the snapshot 2024-08-18T13:41:44.949 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/32 file after the snapshot 2024-08-18T13:41:44.952 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/33 file after the snapshot 2024-08-18T13:41:44.957 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/34 file after the snapshot 2024-08-18T13:41:44.960 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/35 file after the snapshot 2024-08-18T13:41:44.964 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/36 file after the snapshot 2024-08-18T13:41:44.966 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/37 file after the snapshot 2024-08-18T13:41:44.970 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/38 file after the snapshot 2024-08-18T13:41:44.973 INFO:tasks.workunit.client.0.smithi118.stdout: create 210/39 file after the snapshot 2024-08-18T13:41:44.976 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/.snap/snap-subdir-test 2024-08-18T13:41:45.004 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/30 file after the snapshot 2024-08-18T13:41:45.008 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/31 file after the snapshot 2024-08-18T13:41:45.012 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/32 file after the snapshot 2024-08-18T13:41:45.017 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/33 file after the snapshot 2024-08-18T13:41:45.021 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/34 file after the snapshot 2024-08-18T13:41:45.024 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/35 file after the snapshot 2024-08-18T13:41:45.028 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/36 file after the snapshot 2024-08-18T13:41:45.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/37 file after the snapshot 2024-08-18T13:41:45.034 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/38 file after the snapshot 2024-08-18T13:41:45.036 INFO:tasks.workunit.client.0.smithi118.stdout: create 211/39 file after the snapshot 2024-08-18T13:41:45.039 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/.snap/snap-subdir-test 2024-08-18T13:41:45.073 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/30 file after the snapshot 2024-08-18T13:41:45.077 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/31 file after the snapshot 2024-08-18T13:41:45.082 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/32 file after the snapshot 2024-08-18T13:41:45.087 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/33 file after the snapshot 2024-08-18T13:41:45.091 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/34 file after the snapshot 2024-08-18T13:41:45.094 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/35 file after the snapshot 2024-08-18T13:41:45.098 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/36 file after the snapshot 2024-08-18T13:41:45.103 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/37 file after the snapshot 2024-08-18T13:41:45.105 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/38 file after the snapshot 2024-08-18T13:41:45.108 INFO:tasks.workunit.client.0.smithi118.stdout: create 212/39 file after the snapshot 2024-08-18T13:41:45.112 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/.snap/snap-subdir-test 2024-08-18T13:41:45.141 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/30 file after the snapshot 2024-08-18T13:41:45.144 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/31 file after the snapshot 2024-08-18T13:41:45.149 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/32 file after the snapshot 2024-08-18T13:41:45.154 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/33 file after the snapshot 2024-08-18T13:41:45.158 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/34 file after the snapshot 2024-08-18T13:41:45.160 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/35 file after the snapshot 2024-08-18T13:41:45.163 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/36 file after the snapshot 2024-08-18T13:41:45.166 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/37 file after the snapshot 2024-08-18T13:41:45.169 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/38 file after the snapshot 2024-08-18T13:41:45.172 INFO:tasks.workunit.client.0.smithi118.stdout: create 213/39 file after the snapshot 2024-08-18T13:41:45.175 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/.snap/snap-subdir-test 2024-08-18T13:41:45.203 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/30 file after the snapshot 2024-08-18T13:41:45.209 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/31 file after the snapshot 2024-08-18T13:41:45.211 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/32 file after the snapshot 2024-08-18T13:41:45.215 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/33 file after the snapshot 2024-08-18T13:41:45.218 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/34 file after the snapshot 2024-08-18T13:41:45.222 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/35 file after the snapshot 2024-08-18T13:41:45.225 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/36 file after the snapshot 2024-08-18T13:41:45.228 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/37 file after the snapshot 2024-08-18T13:41:45.231 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/38 file after the snapshot 2024-08-18T13:41:45.235 INFO:tasks.workunit.client.0.smithi118.stdout: create 214/39 file after the snapshot 2024-08-18T13:41:45.237 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/.snap/snap-subdir-test 2024-08-18T13:41:45.264 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/30 file after the snapshot 2024-08-18T13:41:45.268 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/31 file after the snapshot 2024-08-18T13:41:45.272 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/32 file after the snapshot 2024-08-18T13:41:45.276 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/33 file after the snapshot 2024-08-18T13:41:45.280 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/34 file after the snapshot 2024-08-18T13:41:45.282 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/35 file after the snapshot 2024-08-18T13:41:45.285 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/36 file after the snapshot 2024-08-18T13:41:45.288 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/37 file after the snapshot 2024-08-18T13:41:45.292 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/38 file after the snapshot 2024-08-18T13:41:45.295 INFO:tasks.workunit.client.0.smithi118.stdout: create 215/39 file after the snapshot 2024-08-18T13:41:45.297 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/.snap/snap-subdir-test 2024-08-18T13:41:45.326 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/30 file after the snapshot 2024-08-18T13:41:45.330 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/31 file after the snapshot 2024-08-18T13:41:45.334 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/32 file after the snapshot 2024-08-18T13:41:45.337 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/33 file after the snapshot 2024-08-18T13:41:45.341 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/34 file after the snapshot 2024-08-18T13:41:45.345 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/35 file after the snapshot 2024-08-18T13:41:45.348 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/36 file after the snapshot 2024-08-18T13:41:45.351 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/37 file after the snapshot 2024-08-18T13:41:45.354 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/38 file after the snapshot 2024-08-18T13:41:45.356 INFO:tasks.workunit.client.0.smithi118.stdout: create 216/39 file after the snapshot 2024-08-18T13:41:45.359 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/.snap/snap-subdir-test 2024-08-18T13:41:45.391 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/30 file after the snapshot 2024-08-18T13:41:45.396 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/31 file after the snapshot 2024-08-18T13:41:45.400 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/32 file after the snapshot 2024-08-18T13:41:45.404 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/33 file after the snapshot 2024-08-18T13:41:45.408 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/34 file after the snapshot 2024-08-18T13:41:45.415 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/35 file after the snapshot 2024-08-18T13:41:45.415 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/36 file after the snapshot 2024-08-18T13:41:45.418 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/37 file after the snapshot 2024-08-18T13:41:45.422 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/38 file after the snapshot 2024-08-18T13:41:45.424 INFO:tasks.workunit.client.0.smithi118.stdout: create 217/39 file after the snapshot 2024-08-18T13:41:45.427 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/.snap/snap-subdir-test 2024-08-18T13:41:45.461 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/30 file after the snapshot 2024-08-18T13:41:45.475 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/31 file after the snapshot 2024-08-18T13:41:45.485 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/32 file after the snapshot 2024-08-18T13:41:45.487 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/33 file after the snapshot 2024-08-18T13:41:45.491 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/34 file after the snapshot 2024-08-18T13:41:45.495 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/35 file after the snapshot 2024-08-18T13:41:45.497 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/36 file after the snapshot 2024-08-18T13:41:45.500 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/37 file after the snapshot 2024-08-18T13:41:45.503 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/38 file after the snapshot 2024-08-18T13:41:45.506 INFO:tasks.workunit.client.0.smithi118.stdout: create 218/39 file after the snapshot 2024-08-18T13:41:45.509 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/.snap/snap-subdir-test 2024-08-18T13:41:45.535 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/30 file after the snapshot 2024-08-18T13:41:45.539 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/31 file after the snapshot 2024-08-18T13:41:45.543 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/32 file after the snapshot 2024-08-18T13:41:45.546 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/33 file after the snapshot 2024-08-18T13:41:45.551 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/34 file after the snapshot 2024-08-18T13:41:45.553 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/35 file after the snapshot 2024-08-18T13:41:45.557 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/36 file after the snapshot 2024-08-18T13:41:45.560 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/37 file after the snapshot 2024-08-18T13:41:45.563 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/38 file after the snapshot 2024-08-18T13:41:45.567 INFO:tasks.workunit.client.0.smithi118.stdout: create 219/39 file after the snapshot 2024-08-18T13:41:45.570 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/.snap/snap-subdir-test 2024-08-18T13:41:45.604 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/30 file after the snapshot 2024-08-18T13:41:45.608 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/31 file after the snapshot 2024-08-18T13:41:45.613 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/32 file after the snapshot 2024-08-18T13:41:45.618 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/33 file after the snapshot 2024-08-18T13:41:45.622 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/34 file after the snapshot 2024-08-18T13:41:45.626 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/35 file after the snapshot 2024-08-18T13:41:45.629 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/36 file after the snapshot 2024-08-18T13:41:45.632 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/37 file after the snapshot 2024-08-18T13:41:45.635 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/38 file after the snapshot 2024-08-18T13:41:45.638 INFO:tasks.workunit.client.0.smithi118.stdout: create 220/39 file after the snapshot 2024-08-18T13:41:45.640 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/.snap/snap-subdir-test 2024-08-18T13:41:45.669 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/30 file after the snapshot 2024-08-18T13:41:45.672 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/31 file after the snapshot 2024-08-18T13:41:45.676 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/32 file after the snapshot 2024-08-18T13:41:45.681 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/33 file after the snapshot 2024-08-18T13:41:45.684 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/34 file after the snapshot 2024-08-18T13:41:45.687 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/35 file after the snapshot 2024-08-18T13:41:45.690 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/36 file after the snapshot 2024-08-18T13:41:45.694 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/37 file after the snapshot 2024-08-18T13:41:45.697 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/38 file after the snapshot 2024-08-18T13:41:45.699 INFO:tasks.workunit.client.0.smithi118.stdout: create 221/39 file after the snapshot 2024-08-18T13:41:45.702 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/.snap/snap-subdir-test 2024-08-18T13:41:45.734 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/30 file after the snapshot 2024-08-18T13:41:45.789 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/31 file after the snapshot 2024-08-18T13:41:45.794 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/32 file after the snapshot 2024-08-18T13:41:45.799 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/33 file after the snapshot 2024-08-18T13:41:45.803 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/34 file after the snapshot 2024-08-18T13:41:45.807 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/35 file after the snapshot 2024-08-18T13:41:45.810 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/36 file after the snapshot 2024-08-18T13:41:45.813 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/37 file after the snapshot 2024-08-18T13:41:45.816 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/38 file after the snapshot 2024-08-18T13:41:45.819 INFO:tasks.workunit.client.0.smithi118.stdout: create 222/39 file after the snapshot 2024-08-18T13:41:45.822 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/.snap/snap-subdir-test 2024-08-18T13:41:45.903 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/30 file after the snapshot 2024-08-18T13:41:45.907 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/31 file after the snapshot 2024-08-18T13:41:45.911 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/32 file after the snapshot 2024-08-18T13:41:45.914 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/33 file after the snapshot 2024-08-18T13:41:45.918 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/34 file after the snapshot 2024-08-18T13:41:45.921 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/35 file after the snapshot 2024-08-18T13:41:45.924 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/36 file after the snapshot 2024-08-18T13:41:45.927 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/37 file after the snapshot 2024-08-18T13:41:45.930 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/38 file after the snapshot 2024-08-18T13:41:45.932 INFO:tasks.workunit.client.0.smithi118.stdout: create 223/39 file after the snapshot 2024-08-18T13:41:45.935 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/.snap/snap-subdir-test 2024-08-18T13:41:45.959 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/30 file after the snapshot 2024-08-18T13:41:45.963 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/31 file after the snapshot 2024-08-18T13:41:45.967 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/32 file after the snapshot 2024-08-18T13:41:45.971 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/33 file after the snapshot 2024-08-18T13:41:45.976 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/34 file after the snapshot 2024-08-18T13:41:45.979 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/35 file after the snapshot 2024-08-18T13:41:45.981 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/36 file after the snapshot 2024-08-18T13:41:45.984 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/37 file after the snapshot 2024-08-18T13:41:45.987 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/38 file after the snapshot 2024-08-18T13:41:45.989 INFO:tasks.workunit.client.0.smithi118.stdout: create 224/39 file after the snapshot 2024-08-18T13:41:45.992 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/.snap/snap-subdir-test 2024-08-18T13:41:46.019 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/30 file after the snapshot 2024-08-18T13:41:46.025 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/31 file after the snapshot 2024-08-18T13:41:46.027 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/32 file after the snapshot 2024-08-18T13:41:46.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/33 file after the snapshot 2024-08-18T13:41:46.034 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/34 file after the snapshot 2024-08-18T13:41:46.037 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/35 file after the snapshot 2024-08-18T13:41:46.041 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/36 file after the snapshot 2024-08-18T13:41:46.044 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/37 file after the snapshot 2024-08-18T13:41:46.047 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/38 file after the snapshot 2024-08-18T13:41:46.050 INFO:tasks.workunit.client.0.smithi118.stdout: create 225/39 file after the snapshot 2024-08-18T13:41:46.054 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/.snap/snap-subdir-test 2024-08-18T13:41:46.080 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/30 file after the snapshot 2024-08-18T13:41:46.083 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/31 file after the snapshot 2024-08-18T13:41:46.087 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/32 file after the snapshot 2024-08-18T13:41:46.091 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/33 file after the snapshot 2024-08-18T13:41:46.094 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/34 file after the snapshot 2024-08-18T13:41:46.098 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/35 file after the snapshot 2024-08-18T13:41:46.101 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/36 file after the snapshot 2024-08-18T13:41:46.105 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/37 file after the snapshot 2024-08-18T13:41:46.108 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/38 file after the snapshot 2024-08-18T13:41:46.111 INFO:tasks.workunit.client.0.smithi118.stdout: create 226/39 file after the snapshot 2024-08-18T13:41:46.114 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/.snap/snap-subdir-test 2024-08-18T13:41:46.139 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/30 file after the snapshot 2024-08-18T13:41:46.143 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/31 file after the snapshot 2024-08-18T13:41:46.146 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/32 file after the snapshot 2024-08-18T13:41:46.150 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/33 file after the snapshot 2024-08-18T13:41:46.153 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/34 file after the snapshot 2024-08-18T13:41:46.157 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/35 file after the snapshot 2024-08-18T13:41:46.159 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/36 file after the snapshot 2024-08-18T13:41:46.162 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/37 file after the snapshot 2024-08-18T13:41:46.165 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/38 file after the snapshot 2024-08-18T13:41:46.168 INFO:tasks.workunit.client.0.smithi118.stdout: create 227/39 file after the snapshot 2024-08-18T13:41:46.171 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/.snap/snap-subdir-test 2024-08-18T13:41:46.200 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/30 file after the snapshot 2024-08-18T13:41:46.204 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/31 file after the snapshot 2024-08-18T13:41:46.208 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/32 file after the snapshot 2024-08-18T13:41:46.212 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/33 file after the snapshot 2024-08-18T13:41:46.217 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/34 file after the snapshot 2024-08-18T13:41:46.219 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/35 file after the snapshot 2024-08-18T13:41:46.222 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/36 file after the snapshot 2024-08-18T13:41:46.225 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/37 file after the snapshot 2024-08-18T13:41:46.231 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/38 file after the snapshot 2024-08-18T13:41:46.231 INFO:tasks.workunit.client.0.smithi118.stdout: create 228/39 file after the snapshot 2024-08-18T13:41:46.234 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/.snap/snap-subdir-test 2024-08-18T13:41:46.261 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/30 file after the snapshot 2024-08-18T13:41:46.264 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/31 file after the snapshot 2024-08-18T13:41:46.269 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/32 file after the snapshot 2024-08-18T13:41:46.273 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/33 file after the snapshot 2024-08-18T13:41:46.276 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/34 file after the snapshot 2024-08-18T13:41:46.280 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/35 file after the snapshot 2024-08-18T13:41:46.283 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/36 file after the snapshot 2024-08-18T13:41:46.286 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/37 file after the snapshot 2024-08-18T13:41:46.290 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/38 file after the snapshot 2024-08-18T13:41:46.292 INFO:tasks.workunit.client.0.smithi118.stdout: create 229/39 file after the snapshot 2024-08-18T13:41:46.296 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/.snap/snap-subdir-test 2024-08-18T13:41:46.329 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/30 file after the snapshot 2024-08-18T13:41:46.333 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/31 file after the snapshot 2024-08-18T13:41:46.338 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/32 file after the snapshot 2024-08-18T13:41:46.342 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/33 file after the snapshot 2024-08-18T13:41:46.346 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/34 file after the snapshot 2024-08-18T13:41:46.350 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/35 file after the snapshot 2024-08-18T13:41:46.353 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/36 file after the snapshot 2024-08-18T13:41:46.357 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/37 file after the snapshot 2024-08-18T13:41:46.360 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/38 file after the snapshot 2024-08-18T13:41:46.363 INFO:tasks.workunit.client.0.smithi118.stdout: create 230/39 file after the snapshot 2024-08-18T13:41:46.366 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/.snap/snap-subdir-test 2024-08-18T13:41:46.391 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/30 file after the snapshot 2024-08-18T13:41:46.394 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/31 file after the snapshot 2024-08-18T13:41:46.398 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/32 file after the snapshot 2024-08-18T13:41:46.403 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/33 file after the snapshot 2024-08-18T13:41:46.407 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/34 file after the snapshot 2024-08-18T13:41:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/35 file after the snapshot 2024-08-18T13:41:46.414 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/36 file after the snapshot 2024-08-18T13:41:46.416 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/37 file after the snapshot 2024-08-18T13:41:46.420 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/38 file after the snapshot 2024-08-18T13:41:46.422 INFO:tasks.workunit.client.0.smithi118.stdout: create 231/39 file after the snapshot 2024-08-18T13:41:46.425 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/.snap/snap-subdir-test 2024-08-18T13:41:46.449 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/30 file after the snapshot 2024-08-18T13:41:46.452 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/31 file after the snapshot 2024-08-18T13:41:46.455 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/32 file after the snapshot 2024-08-18T13:41:46.460 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/33 file after the snapshot 2024-08-18T13:41:46.463 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/34 file after the snapshot 2024-08-18T13:41:46.466 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/35 file after the snapshot 2024-08-18T13:41:46.469 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/36 file after the snapshot 2024-08-18T13:41:46.473 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/37 file after the snapshot 2024-08-18T13:41:46.475 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/38 file after the snapshot 2024-08-18T13:41:46.479 INFO:tasks.workunit.client.0.smithi118.stdout: create 232/39 file after the snapshot 2024-08-18T13:41:46.482 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/.snap/snap-subdir-test 2024-08-18T13:41:46.506 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/30 file after the snapshot 2024-08-18T13:41:46.508 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/31 file after the snapshot 2024-08-18T13:41:46.512 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/32 file after the snapshot 2024-08-18T13:41:46.516 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/33 file after the snapshot 2024-08-18T13:41:46.521 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/34 file after the snapshot 2024-08-18T13:41:46.524 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/35 file after the snapshot 2024-08-18T13:41:46.528 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/36 file after the snapshot 2024-08-18T13:41:46.530 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/37 file after the snapshot 2024-08-18T13:41:46.534 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/38 file after the snapshot 2024-08-18T13:41:46.537 INFO:tasks.workunit.client.0.smithi118.stdout: create 233/39 file after the snapshot 2024-08-18T13:41:46.540 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/.snap/snap-subdir-test 2024-08-18T13:41:46.564 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/30 file after the snapshot 2024-08-18T13:41:46.568 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/31 file after the snapshot 2024-08-18T13:41:46.573 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/32 file after the snapshot 2024-08-18T13:41:46.576 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/33 file after the snapshot 2024-08-18T13:41:46.580 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/34 file after the snapshot 2024-08-18T13:41:46.584 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/35 file after the snapshot 2024-08-18T13:41:46.587 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/36 file after the snapshot 2024-08-18T13:41:46.590 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/37 file after the snapshot 2024-08-18T13:41:46.594 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/38 file after the snapshot 2024-08-18T13:41:46.597 INFO:tasks.workunit.client.0.smithi118.stdout: create 234/39 file after the snapshot 2024-08-18T13:41:46.599 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/.snap/snap-subdir-test 2024-08-18T13:41:46.625 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/30 file after the snapshot 2024-08-18T13:41:46.628 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/31 file after the snapshot 2024-08-18T13:41:46.632 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/32 file after the snapshot 2024-08-18T13:41:46.636 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/33 file after the snapshot 2024-08-18T13:41:46.642 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/34 file after the snapshot 2024-08-18T13:41:46.643 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/35 file after the snapshot 2024-08-18T13:41:46.647 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/36 file after the snapshot 2024-08-18T13:41:46.650 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/37 file after the snapshot 2024-08-18T13:41:46.652 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/38 file after the snapshot 2024-08-18T13:41:46.656 INFO:tasks.workunit.client.0.smithi118.stdout: create 235/39 file after the snapshot 2024-08-18T13:41:46.658 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/.snap/snap-subdir-test 2024-08-18T13:41:46.684 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/30 file after the snapshot 2024-08-18T13:41:46.688 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/31 file after the snapshot 2024-08-18T13:41:46.692 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/32 file after the snapshot 2024-08-18T13:41:46.696 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/33 file after the snapshot 2024-08-18T13:41:46.700 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/34 file after the snapshot 2024-08-18T13:41:46.703 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/35 file after the snapshot 2024-08-18T13:41:46.706 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/36 file after the snapshot 2024-08-18T13:41:46.709 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/37 file after the snapshot 2024-08-18T13:41:46.712 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/38 file after the snapshot 2024-08-18T13:41:46.715 INFO:tasks.workunit.client.0.smithi118.stdout: create 236/39 file after the snapshot 2024-08-18T13:41:46.718 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/.snap/snap-subdir-test 2024-08-18T13:41:46.761 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/30 file after the snapshot 2024-08-18T13:41:46.805 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/31 file after the snapshot 2024-08-18T13:41:46.809 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/32 file after the snapshot 2024-08-18T13:41:46.813 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/33 file after the snapshot 2024-08-18T13:41:46.817 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/34 file after the snapshot 2024-08-18T13:41:46.820 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/35 file after the snapshot 2024-08-18T13:41:46.823 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/36 file after the snapshot 2024-08-18T13:41:46.825 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/37 file after the snapshot 2024-08-18T13:41:46.828 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/38 file after the snapshot 2024-08-18T13:41:46.831 INFO:tasks.workunit.client.0.smithi118.stdout: create 237/39 file after the snapshot 2024-08-18T13:41:46.834 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/.snap/snap-subdir-test 2024-08-18T13:41:46.888 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/30 file after the snapshot 2024-08-18T13:41:46.892 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/31 file after the snapshot 2024-08-18T13:41:46.896 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/32 file after the snapshot 2024-08-18T13:41:46.900 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/33 file after the snapshot 2024-08-18T13:41:46.904 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/34 file after the snapshot 2024-08-18T13:41:46.907 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/35 file after the snapshot 2024-08-18T13:41:46.910 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/36 file after the snapshot 2024-08-18T13:41:46.912 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/37 file after the snapshot 2024-08-18T13:41:46.915 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/38 file after the snapshot 2024-08-18T13:41:46.918 INFO:tasks.workunit.client.0.smithi118.stdout: create 238/39 file after the snapshot 2024-08-18T13:41:46.921 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/.snap/snap-subdir-test 2024-08-18T13:41:46.952 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/30 file after the snapshot 2024-08-18T13:41:46.956 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/31 file after the snapshot 2024-08-18T13:41:46.960 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/32 file after the snapshot 2024-08-18T13:41:46.965 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/33 file after the snapshot 2024-08-18T13:41:46.969 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/34 file after the snapshot 2024-08-18T13:41:46.972 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/35 file after the snapshot 2024-08-18T13:41:46.974 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/36 file after the snapshot 2024-08-18T13:41:46.978 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/37 file after the snapshot 2024-08-18T13:41:46.982 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/38 file after the snapshot 2024-08-18T13:41:46.985 INFO:tasks.workunit.client.0.smithi118.stdout: create 239/39 file after the snapshot 2024-08-18T13:41:46.989 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/.snap/snap-subdir-test 2024-08-18T13:41:47.018 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/30 file after the snapshot 2024-08-18T13:41:47.021 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/31 file after the snapshot 2024-08-18T13:41:47.025 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/32 file after the snapshot 2024-08-18T13:41:47.028 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/33 file after the snapshot 2024-08-18T13:41:47.034 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/34 file after the snapshot 2024-08-18T13:41:47.038 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/35 file after the snapshot 2024-08-18T13:41:47.040 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/36 file after the snapshot 2024-08-18T13:41:47.043 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/37 file after the snapshot 2024-08-18T13:41:47.046 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/38 file after the snapshot 2024-08-18T13:41:47.049 INFO:tasks.workunit.client.0.smithi118.stdout: create 240/39 file after the snapshot 2024-08-18T13:41:47.052 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/.snap/snap-subdir-test 2024-08-18T13:41:47.083 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/30 file after the snapshot 2024-08-18T13:41:47.087 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/31 file after the snapshot 2024-08-18T13:41:47.092 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/32 file after the snapshot 2024-08-18T13:41:47.096 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/33 file after the snapshot 2024-08-18T13:41:47.099 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/34 file after the snapshot 2024-08-18T13:41:47.103 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/35 file after the snapshot 2024-08-18T13:41:47.106 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/36 file after the snapshot 2024-08-18T13:41:47.109 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/37 file after the snapshot 2024-08-18T13:41:47.112 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/38 file after the snapshot 2024-08-18T13:41:47.114 INFO:tasks.workunit.client.0.smithi118.stdout: create 241/39 file after the snapshot 2024-08-18T13:41:47.117 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/.snap/snap-subdir-test 2024-08-18T13:41:47.143 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/30 file after the snapshot 2024-08-18T13:41:47.146 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/31 file after the snapshot 2024-08-18T13:41:47.151 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/32 file after the snapshot 2024-08-18T13:41:47.155 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/33 file after the snapshot 2024-08-18T13:41:47.159 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/34 file after the snapshot 2024-08-18T13:41:47.163 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/35 file after the snapshot 2024-08-18T13:41:47.165 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/36 file after the snapshot 2024-08-18T13:41:47.168 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/37 file after the snapshot 2024-08-18T13:41:47.172 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/38 file after the snapshot 2024-08-18T13:41:47.173 INFO:tasks.workunit.client.0.smithi118.stdout: create 242/39 file after the snapshot 2024-08-18T13:41:47.176 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/.snap/snap-subdir-test 2024-08-18T13:41:47.209 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/30 file after the snapshot 2024-08-18T13:41:47.214 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/31 file after the snapshot 2024-08-18T13:41:47.219 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/32 file after the snapshot 2024-08-18T13:41:47.223 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/33 file after the snapshot 2024-08-18T13:41:47.226 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/34 file after the snapshot 2024-08-18T13:41:47.230 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/35 file after the snapshot 2024-08-18T13:41:47.233 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/36 file after the snapshot 2024-08-18T13:41:47.236 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/37 file after the snapshot 2024-08-18T13:41:47.239 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/38 file after the snapshot 2024-08-18T13:41:47.242 INFO:tasks.workunit.client.0.smithi118.stdout: create 243/39 file after the snapshot 2024-08-18T13:41:47.245 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/.snap/snap-subdir-test 2024-08-18T13:41:47.275 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/30 file after the snapshot 2024-08-18T13:41:47.280 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/31 file after the snapshot 2024-08-18T13:41:47.285 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/32 file after the snapshot 2024-08-18T13:41:47.289 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/33 file after the snapshot 2024-08-18T13:41:47.293 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/34 file after the snapshot 2024-08-18T13:41:47.296 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/35 file after the snapshot 2024-08-18T13:41:47.299 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/36 file after the snapshot 2024-08-18T13:41:47.303 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/37 file after the snapshot 2024-08-18T13:41:47.305 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/38 file after the snapshot 2024-08-18T13:41:47.308 INFO:tasks.workunit.client.0.smithi118.stdout: create 244/39 file after the snapshot 2024-08-18T13:41:47.311 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/.snap/snap-subdir-test 2024-08-18T13:41:47.339 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/30 file after the snapshot 2024-08-18T13:41:47.343 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/31 file after the snapshot 2024-08-18T13:41:47.348 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/32 file after the snapshot 2024-08-18T13:41:47.353 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/33 file after the snapshot 2024-08-18T13:41:47.357 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/34 file after the snapshot 2024-08-18T13:41:47.361 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/35 file after the snapshot 2024-08-18T13:41:47.364 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/36 file after the snapshot 2024-08-18T13:41:47.367 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/37 file after the snapshot 2024-08-18T13:41:47.370 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/38 file after the snapshot 2024-08-18T13:41:47.374 INFO:tasks.workunit.client.0.smithi118.stdout: create 245/39 file after the snapshot 2024-08-18T13:41:47.378 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/.snap/snap-subdir-test 2024-08-18T13:41:47.403 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/30 file after the snapshot 2024-08-18T13:41:47.406 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/31 file after the snapshot 2024-08-18T13:41:47.410 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/32 file after the snapshot 2024-08-18T13:41:47.414 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/33 file after the snapshot 2024-08-18T13:41:47.418 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/34 file after the snapshot 2024-08-18T13:41:47.422 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/35 file after the snapshot 2024-08-18T13:41:47.425 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/36 file after the snapshot 2024-08-18T13:41:47.428 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/37 file after the snapshot 2024-08-18T13:41:47.431 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/38 file after the snapshot 2024-08-18T13:41:47.434 INFO:tasks.workunit.client.0.smithi118.stdout: create 246/39 file after the snapshot 2024-08-18T13:41:47.438 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/.snap/snap-subdir-test 2024-08-18T13:41:47.465 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/30 file after the snapshot 2024-08-18T13:41:47.469 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/31 file after the snapshot 2024-08-18T13:41:47.472 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/32 file after the snapshot 2024-08-18T13:41:47.475 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/33 file after the snapshot 2024-08-18T13:41:47.479 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/34 file after the snapshot 2024-08-18T13:41:47.482 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/35 file after the snapshot 2024-08-18T13:41:47.486 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/36 file after the snapshot 2024-08-18T13:41:47.488 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/37 file after the snapshot 2024-08-18T13:41:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/38 file after the snapshot 2024-08-18T13:41:47.494 INFO:tasks.workunit.client.0.smithi118.stdout: create 247/39 file after the snapshot 2024-08-18T13:41:47.497 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/.snap/snap-subdir-test 2024-08-18T13:41:47.521 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/30 file after the snapshot 2024-08-18T13:41:47.525 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/31 file after the snapshot 2024-08-18T13:41:47.529 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/32 file after the snapshot 2024-08-18T13:41:47.534 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/33 file after the snapshot 2024-08-18T13:41:47.537 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/34 file after the snapshot 2024-08-18T13:41:47.540 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/35 file after the snapshot 2024-08-18T13:41:47.543 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/36 file after the snapshot 2024-08-18T13:41:47.546 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/37 file after the snapshot 2024-08-18T13:41:47.549 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/38 file after the snapshot 2024-08-18T13:41:47.552 INFO:tasks.workunit.client.0.smithi118.stdout: create 248/39 file after the snapshot 2024-08-18T13:41:47.555 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/.snap/snap-subdir-test 2024-08-18T13:41:47.581 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/30 file after the snapshot 2024-08-18T13:41:47.585 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/31 file after the snapshot 2024-08-18T13:41:47.589 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/32 file after the snapshot 2024-08-18T13:41:47.594 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/33 file after the snapshot 2024-08-18T13:41:47.597 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/34 file after the snapshot 2024-08-18T13:41:47.602 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/35 file after the snapshot 2024-08-18T13:41:47.604 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/36 file after the snapshot 2024-08-18T13:41:47.607 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/37 file after the snapshot 2024-08-18T13:41:47.610 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/38 file after the snapshot 2024-08-18T13:41:47.613 INFO:tasks.workunit.client.0.smithi118.stdout: create 249/39 file after the snapshot 2024-08-18T13:41:47.616 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/.snap/snap-subdir-test 2024-08-18T13:41:47.642 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/30 file after the snapshot 2024-08-18T13:41:47.645 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/31 file after the snapshot 2024-08-18T13:41:47.648 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/32 file after the snapshot 2024-08-18T13:41:47.652 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/33 file after the snapshot 2024-08-18T13:41:47.656 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/34 file after the snapshot 2024-08-18T13:41:47.659 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/35 file after the snapshot 2024-08-18T13:41:47.662 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/36 file after the snapshot 2024-08-18T13:41:47.666 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/37 file after the snapshot 2024-08-18T13:41:47.668 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/38 file after the snapshot 2024-08-18T13:41:47.671 INFO:tasks.workunit.client.0.smithi118.stdout: create 250/39 file after the snapshot 2024-08-18T13:41:47.674 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/.snap/snap-subdir-test 2024-08-18T13:41:47.701 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/30 file after the snapshot 2024-08-18T13:41:47.705 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/31 file after the snapshot 2024-08-18T13:41:47.707 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/32 file after the snapshot 2024-08-18T13:41:47.711 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/33 file after the snapshot 2024-08-18T13:41:47.714 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/34 file after the snapshot 2024-08-18T13:41:47.718 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/35 file after the snapshot 2024-08-18T13:41:47.721 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/36 file after the snapshot 2024-08-18T13:41:47.725 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/37 file after the snapshot 2024-08-18T13:41:47.727 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/38 file after the snapshot 2024-08-18T13:41:47.730 INFO:tasks.workunit.client.0.smithi118.stdout: create 251/39 file after the snapshot 2024-08-18T13:41:47.733 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/.snap/snap-subdir-test 2024-08-18T13:41:47.784 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/30 file after the snapshot 2024-08-18T13:41:47.819 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/31 file after the snapshot 2024-08-18T13:41:47.823 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/32 file after the snapshot 2024-08-18T13:41:47.828 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/33 file after the snapshot 2024-08-18T13:41:47.832 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/34 file after the snapshot 2024-08-18T13:41:47.834 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/35 file after the snapshot 2024-08-18T13:41:47.837 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/36 file after the snapshot 2024-08-18T13:41:47.840 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/37 file after the snapshot 2024-08-18T13:41:47.843 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/38 file after the snapshot 2024-08-18T13:41:47.845 INFO:tasks.workunit.client.0.smithi118.stdout: create 252/39 file after the snapshot 2024-08-18T13:41:47.848 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/.snap/snap-subdir-test 2024-08-18T13:41:47.897 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/30 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/31 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/32 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/33 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/34 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/35 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/36 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/37 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/38 file after the snapshot 2024-08-18T13:41:48.029 INFO:tasks.workunit.client.0.smithi118.stdout: create 253/39 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/.snap/snap-subdir-test 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/30 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/31 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/32 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/33 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/34 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/35 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/36 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/37 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/38 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 254/39 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/.snap/snap-subdir-test 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/30 file after the snapshot 2024-08-18T13:41:48.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/31 file after the snapshot 2024-08-18T13:41:48.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/32 file after the snapshot 2024-08-18T13:41:48.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/33 file after the snapshot 2024-08-18T13:41:48.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/34 file after the snapshot 2024-08-18T13:41:48.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/35 file after the snapshot 2024-08-18T13:41:48.031 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/36 file after the snapshot 2024-08-18T13:41:48.033 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/37 file after the snapshot 2024-08-18T13:41:48.036 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/38 file after the snapshot 2024-08-18T13:41:48.039 INFO:tasks.workunit.client.0.smithi118.stdout: create 255/39 file after the snapshot 2024-08-18T13:41:48.043 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/.snap/snap-subdir-test 2024-08-18T13:41:48.069 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/30 file after the snapshot 2024-08-18T13:41:48.073 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/31 file after the snapshot 2024-08-18T13:41:48.077 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/32 file after the snapshot 2024-08-18T13:41:48.082 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/33 file after the snapshot 2024-08-18T13:41:48.086 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/34 file after the snapshot 2024-08-18T13:41:48.088 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/35 file after the snapshot 2024-08-18T13:41:48.093 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/36 file after the snapshot 2024-08-18T13:41:48.096 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/37 file after the snapshot 2024-08-18T13:41:48.099 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/38 file after the snapshot 2024-08-18T13:41:48.101 INFO:tasks.workunit.client.0.smithi118.stdout: create 256/39 file after the snapshot 2024-08-18T13:41:48.104 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/.snap/snap-subdir-test 2024-08-18T13:41:48.133 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/30 file after the snapshot 2024-08-18T13:41:48.136 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/31 file after the snapshot 2024-08-18T13:41:48.140 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/32 file after the snapshot 2024-08-18T13:41:48.144 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/33 file after the snapshot 2024-08-18T13:41:48.147 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/34 file after the snapshot 2024-08-18T13:41:48.150 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/35 file after the snapshot 2024-08-18T13:41:48.153 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/36 file after the snapshot 2024-08-18T13:41:48.156 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/37 file after the snapshot 2024-08-18T13:41:48.159 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/38 file after the snapshot 2024-08-18T13:41:48.162 INFO:tasks.workunit.client.0.smithi118.stdout: create 257/39 file after the snapshot 2024-08-18T13:41:48.165 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/.snap/snap-subdir-test 2024-08-18T13:41:48.193 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/30 file after the snapshot 2024-08-18T13:41:48.197 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/31 file after the snapshot 2024-08-18T13:41:48.201 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/32 file after the snapshot 2024-08-18T13:41:48.206 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/33 file after the snapshot 2024-08-18T13:41:48.210 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/34 file after the snapshot 2024-08-18T13:41:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/35 file after the snapshot 2024-08-18T13:41:48.216 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/36 file after the snapshot 2024-08-18T13:41:48.219 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/37 file after the snapshot 2024-08-18T13:41:48.222 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/38 file after the snapshot 2024-08-18T13:41:48.225 INFO:tasks.workunit.client.0.smithi118.stdout: create 258/39 file after the snapshot 2024-08-18T13:41:48.229 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/.snap/snap-subdir-test 2024-08-18T13:41:48.252 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/30 file after the snapshot 2024-08-18T13:41:48.256 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/31 file after the snapshot 2024-08-18T13:41:48.259 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/32 file after the snapshot 2024-08-18T13:41:48.263 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/33 file after the snapshot 2024-08-18T13:41:48.266 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/34 file after the snapshot 2024-08-18T13:41:48.269 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/35 file after the snapshot 2024-08-18T13:41:48.272 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/36 file after the snapshot 2024-08-18T13:41:48.275 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/37 file after the snapshot 2024-08-18T13:41:48.278 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/38 file after the snapshot 2024-08-18T13:41:48.280 INFO:tasks.workunit.client.0.smithi118.stdout: create 259/39 file after the snapshot 2024-08-18T13:41:48.283 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/.snap/snap-subdir-test 2024-08-18T13:41:48.310 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/30 file after the snapshot 2024-08-18T13:41:48.314 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/31 file after the snapshot 2024-08-18T13:41:48.318 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/32 file after the snapshot 2024-08-18T13:41:48.322 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/33 file after the snapshot 2024-08-18T13:41:48.327 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/34 file after the snapshot 2024-08-18T13:41:48.330 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/35 file after the snapshot 2024-08-18T13:41:48.333 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/36 file after the snapshot 2024-08-18T13:41:48.336 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/37 file after the snapshot 2024-08-18T13:41:48.340 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/38 file after the snapshot 2024-08-18T13:41:48.343 INFO:tasks.workunit.client.0.smithi118.stdout: create 260/39 file after the snapshot 2024-08-18T13:41:48.346 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/.snap/snap-subdir-test 2024-08-18T13:41:48.371 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/30 file after the snapshot 2024-08-18T13:41:48.374 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/31 file after the snapshot 2024-08-18T13:41:48.378 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/32 file after the snapshot 2024-08-18T13:41:48.382 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/33 file after the snapshot 2024-08-18T13:41:48.385 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/34 file after the snapshot 2024-08-18T13:41:48.388 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/35 file after the snapshot 2024-08-18T13:41:48.391 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/36 file after the snapshot 2024-08-18T13:41:48.394 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/37 file after the snapshot 2024-08-18T13:41:48.398 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/38 file after the snapshot 2024-08-18T13:41:48.401 INFO:tasks.workunit.client.0.smithi118.stdout: create 261/39 file after the snapshot 2024-08-18T13:41:48.403 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/.snap/snap-subdir-test 2024-08-18T13:41:48.430 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/30 file after the snapshot 2024-08-18T13:41:48.434 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/31 file after the snapshot 2024-08-18T13:41:48.438 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/32 file after the snapshot 2024-08-18T13:41:48.441 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/33 file after the snapshot 2024-08-18T13:41:48.445 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/34 file after the snapshot 2024-08-18T13:41:48.448 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/35 file after the snapshot 2024-08-18T13:41:48.452 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/36 file after the snapshot 2024-08-18T13:41:48.455 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/37 file after the snapshot 2024-08-18T13:41:48.457 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/38 file after the snapshot 2024-08-18T13:41:48.461 INFO:tasks.workunit.client.0.smithi118.stdout: create 262/39 file after the snapshot 2024-08-18T13:41:48.464 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/.snap/snap-subdir-test 2024-08-18T13:41:48.489 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/30 file after the snapshot 2024-08-18T13:41:48.493 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/31 file after the snapshot 2024-08-18T13:41:48.497 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/32 file after the snapshot 2024-08-18T13:41:48.501 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/33 file after the snapshot 2024-08-18T13:41:48.505 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/34 file after the snapshot 2024-08-18T13:41:48.508 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/35 file after the snapshot 2024-08-18T13:41:48.511 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/36 file after the snapshot 2024-08-18T13:41:48.515 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/37 file after the snapshot 2024-08-18T13:41:48.518 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/38 file after the snapshot 2024-08-18T13:41:48.521 INFO:tasks.workunit.client.0.smithi118.stdout: create 263/39 file after the snapshot 2024-08-18T13:41:48.524 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/.snap/snap-subdir-test 2024-08-18T13:41:48.548 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/30 file after the snapshot 2024-08-18T13:41:48.551 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/31 file after the snapshot 2024-08-18T13:41:48.554 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/32 file after the snapshot 2024-08-18T13:41:48.559 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/33 file after the snapshot 2024-08-18T13:41:48.563 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/34 file after the snapshot 2024-08-18T13:41:48.567 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/35 file after the snapshot 2024-08-18T13:41:48.570 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/36 file after the snapshot 2024-08-18T13:41:48.574 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/37 file after the snapshot 2024-08-18T13:41:48.577 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/38 file after the snapshot 2024-08-18T13:41:48.580 INFO:tasks.workunit.client.0.smithi118.stdout: create 264/39 file after the snapshot 2024-08-18T13:41:48.583 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/.snap/snap-subdir-test 2024-08-18T13:41:48.612 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/30 file after the snapshot 2024-08-18T13:41:48.615 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/31 file after the snapshot 2024-08-18T13:41:48.619 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/32 file after the snapshot 2024-08-18T13:41:48.623 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/33 file after the snapshot 2024-08-18T13:41:48.628 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/34 file after the snapshot 2024-08-18T13:41:48.631 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/35 file after the snapshot 2024-08-18T13:41:48.634 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/36 file after the snapshot 2024-08-18T13:41:48.637 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/37 file after the snapshot 2024-08-18T13:41:48.642 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/38 file after the snapshot 2024-08-18T13:41:48.645 INFO:tasks.workunit.client.0.smithi118.stdout: create 265/39 file after the snapshot 2024-08-18T13:41:48.648 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/.snap/snap-subdir-test 2024-08-18T13:41:48.674 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/30 file after the snapshot 2024-08-18T13:41:48.678 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/31 file after the snapshot 2024-08-18T13:41:48.681 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/32 file after the snapshot 2024-08-18T13:41:48.686 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/33 file after the snapshot 2024-08-18T13:41:48.690 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/34 file after the snapshot 2024-08-18T13:41:48.693 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/35 file after the snapshot 2024-08-18T13:41:48.696 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/36 file after the snapshot 2024-08-18T13:41:48.700 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/37 file after the snapshot 2024-08-18T13:41:48.702 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/38 file after the snapshot 2024-08-18T13:41:48.706 INFO:tasks.workunit.client.0.smithi118.stdout: create 266/39 file after the snapshot 2024-08-18T13:41:48.709 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/.snap/snap-subdir-test 2024-08-18T13:41:48.738 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/30 file after the snapshot 2024-08-18T13:41:48.742 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/31 file after the snapshot 2024-08-18T13:41:48.748 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/32 file after the snapshot 2024-08-18T13:41:48.752 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/33 file after the snapshot 2024-08-18T13:41:48.763 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/34 file after the snapshot 2024-08-18T13:41:48.818 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/35 file after the snapshot 2024-08-18T13:41:48.821 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/36 file after the snapshot 2024-08-18T13:41:48.825 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/37 file after the snapshot 2024-08-18T13:41:48.828 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/38 file after the snapshot 2024-08-18T13:41:48.831 INFO:tasks.workunit.client.0.smithi118.stdout: create 267/39 file after the snapshot 2024-08-18T13:41:48.835 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/.snap/snap-subdir-test 2024-08-18T13:41:48.881 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/30 file after the snapshot 2024-08-18T13:41:48.885 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/31 file after the snapshot 2024-08-18T13:41:48.889 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/32 file after the snapshot 2024-08-18T13:41:48.893 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/33 file after the snapshot 2024-08-18T13:41:48.895 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/34 file after the snapshot 2024-08-18T13:41:48.898 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/35 file after the snapshot 2024-08-18T13:41:48.901 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/36 file after the snapshot 2024-08-18T13:41:48.904 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/37 file after the snapshot 2024-08-18T13:41:48.907 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/38 file after the snapshot 2024-08-18T13:41:48.910 INFO:tasks.workunit.client.0.smithi118.stdout: create 268/39 file after the snapshot 2024-08-18T13:41:48.912 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/.snap/snap-subdir-test 2024-08-18T13:41:48.933 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/30 file after the snapshot 2024-08-18T13:41:48.936 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/31 file after the snapshot 2024-08-18T13:41:48.940 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/32 file after the snapshot 2024-08-18T13:41:48.944 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/33 file after the snapshot 2024-08-18T13:41:48.947 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/34 file after the snapshot 2024-08-18T13:41:48.951 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/35 file after the snapshot 2024-08-18T13:41:48.954 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/36 file after the snapshot 2024-08-18T13:41:48.957 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/37 file after the snapshot 2024-08-18T13:41:48.959 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/38 file after the snapshot 2024-08-18T13:41:48.963 INFO:tasks.workunit.client.0.smithi118.stdout: create 269/39 file after the snapshot 2024-08-18T13:41:48.965 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/.snap/snap-subdir-test 2024-08-18T13:41:48.990 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/30 file after the snapshot 2024-08-18T13:41:48.993 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/31 file after the snapshot 2024-08-18T13:41:48.997 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/32 file after the snapshot 2024-08-18T13:41:49.000 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/33 file after the snapshot 2024-08-18T13:41:49.005 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/34 file after the snapshot 2024-08-18T13:41:49.007 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/35 file after the snapshot 2024-08-18T13:41:49.010 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/36 file after the snapshot 2024-08-18T13:41:49.013 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/37 file after the snapshot 2024-08-18T13:41:49.016 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/38 file after the snapshot 2024-08-18T13:41:49.020 INFO:tasks.workunit.client.0.smithi118.stdout: create 270/39 file after the snapshot 2024-08-18T13:41:49.022 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/.snap/snap-subdir-test 2024-08-18T13:41:49.047 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/30 file after the snapshot 2024-08-18T13:41:49.051 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/31 file after the snapshot 2024-08-18T13:41:49.056 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/32 file after the snapshot 2024-08-18T13:41:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/33 file after the snapshot 2024-08-18T13:41:49.063 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/34 file after the snapshot 2024-08-18T13:41:49.066 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/35 file after the snapshot 2024-08-18T13:41:49.069 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/36 file after the snapshot 2024-08-18T13:41:49.072 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/37 file after the snapshot 2024-08-18T13:41:49.075 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/38 file after the snapshot 2024-08-18T13:41:49.078 INFO:tasks.workunit.client.0.smithi118.stdout: create 271/39 file after the snapshot 2024-08-18T13:41:49.081 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/.snap/snap-subdir-test 2024-08-18T13:41:49.103 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/30 file after the snapshot 2024-08-18T13:41:49.106 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/31 file after the snapshot 2024-08-18T13:41:49.111 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/32 file after the snapshot 2024-08-18T13:41:49.115 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/33 file after the snapshot 2024-08-18T13:41:49.119 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/34 file after the snapshot 2024-08-18T13:41:49.122 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/35 file after the snapshot 2024-08-18T13:41:49.126 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/36 file after the snapshot 2024-08-18T13:41:49.129 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/37 file after the snapshot 2024-08-18T13:41:49.132 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/38 file after the snapshot 2024-08-18T13:41:49.135 INFO:tasks.workunit.client.0.smithi118.stdout: create 272/39 file after the snapshot 2024-08-18T13:41:49.139 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/.snap/snap-subdir-test 2024-08-18T13:41:49.166 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/30 file after the snapshot 2024-08-18T13:41:49.170 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/31 file after the snapshot 2024-08-18T13:41:49.175 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/32 file after the snapshot 2024-08-18T13:41:49.179 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/33 file after the snapshot 2024-08-18T13:41:49.183 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/34 file after the snapshot 2024-08-18T13:41:49.186 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/35 file after the snapshot 2024-08-18T13:41:49.188 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/36 file after the snapshot 2024-08-18T13:41:49.192 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/37 file after the snapshot 2024-08-18T13:41:49.194 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/38 file after the snapshot 2024-08-18T13:41:49.197 INFO:tasks.workunit.client.0.smithi118.stdout: create 273/39 file after the snapshot 2024-08-18T13:41:49.200 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/.snap/snap-subdir-test 2024-08-18T13:41:49.223 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/30 file after the snapshot 2024-08-18T13:41:49.226 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/31 file after the snapshot 2024-08-18T13:41:49.231 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/32 file after the snapshot 2024-08-18T13:41:49.235 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/33 file after the snapshot 2024-08-18T13:41:49.240 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/34 file after the snapshot 2024-08-18T13:41:49.243 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/35 file after the snapshot 2024-08-18T13:41:49.246 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/36 file after the snapshot 2024-08-18T13:41:49.249 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/37 file after the snapshot 2024-08-18T13:41:49.252 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/38 file after the snapshot 2024-08-18T13:41:49.255 INFO:tasks.workunit.client.0.smithi118.stdout: create 274/39 file after the snapshot 2024-08-18T13:41:49.258 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/.snap/snap-subdir-test 2024-08-18T13:41:49.284 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/30 file after the snapshot 2024-08-18T13:41:49.288 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/31 file after the snapshot 2024-08-18T13:41:49.292 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/32 file after the snapshot 2024-08-18T13:41:49.296 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/33 file after the snapshot 2024-08-18T13:41:49.303 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/34 file after the snapshot 2024-08-18T13:41:49.303 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/35 file after the snapshot 2024-08-18T13:41:49.305 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/36 file after the snapshot 2024-08-18T13:41:49.309 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/37 file after the snapshot 2024-08-18T13:41:49.311 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/38 file after the snapshot 2024-08-18T13:41:49.314 INFO:tasks.workunit.client.0.smithi118.stdout: create 275/39 file after the snapshot 2024-08-18T13:41:49.318 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/.snap/snap-subdir-test 2024-08-18T13:41:49.343 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/30 file after the snapshot 2024-08-18T13:41:49.347 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/31 file after the snapshot 2024-08-18T13:41:49.351 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/32 file after the snapshot 2024-08-18T13:41:49.356 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/33 file after the snapshot 2024-08-18T13:41:49.361 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/34 file after the snapshot 2024-08-18T13:41:49.364 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/35 file after the snapshot 2024-08-18T13:41:49.367 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/36 file after the snapshot 2024-08-18T13:41:49.371 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/37 file after the snapshot 2024-08-18T13:41:49.374 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/38 file after the snapshot 2024-08-18T13:41:49.376 INFO:tasks.workunit.client.0.smithi118.stdout: create 276/39 file after the snapshot 2024-08-18T13:41:49.380 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/.snap/snap-subdir-test 2024-08-18T13:41:49.403 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/30 file after the snapshot 2024-08-18T13:41:49.407 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/31 file after the snapshot 2024-08-18T13:41:49.411 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/32 file after the snapshot 2024-08-18T13:41:49.415 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/33 file after the snapshot 2024-08-18T13:41:49.419 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/34 file after the snapshot 2024-08-18T13:41:49.422 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/35 file after the snapshot 2024-08-18T13:41:49.425 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/36 file after the snapshot 2024-08-18T13:41:49.428 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/37 file after the snapshot 2024-08-18T13:41:49.431 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/38 file after the snapshot 2024-08-18T13:41:49.434 INFO:tasks.workunit.client.0.smithi118.stdout: create 277/39 file after the snapshot 2024-08-18T13:41:49.437 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/.snap/snap-subdir-test 2024-08-18T13:41:49.459 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/30 file after the snapshot 2024-08-18T13:41:49.462 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/31 file after the snapshot 2024-08-18T13:41:49.465 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/32 file after the snapshot 2024-08-18T13:41:49.469 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/33 file after the snapshot 2024-08-18T13:41:49.473 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/34 file after the snapshot 2024-08-18T13:41:49.476 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/35 file after the snapshot 2024-08-18T13:41:49.479 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/36 file after the snapshot 2024-08-18T13:41:49.483 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/37 file after the snapshot 2024-08-18T13:41:49.485 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/38 file after the snapshot 2024-08-18T13:41:49.488 INFO:tasks.workunit.client.0.smithi118.stdout: create 278/39 file after the snapshot 2024-08-18T13:41:49.491 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/.snap/snap-subdir-test 2024-08-18T13:41:49.514 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/30 file after the snapshot 2024-08-18T13:41:49.518 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/31 file after the snapshot 2024-08-18T13:41:49.522 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/32 file after the snapshot 2024-08-18T13:41:49.528 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/33 file after the snapshot 2024-08-18T13:41:49.532 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/34 file after the snapshot 2024-08-18T13:41:49.535 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/35 file after the snapshot 2024-08-18T13:41:49.538 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/36 file after the snapshot 2024-08-18T13:41:49.540 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/37 file after the snapshot 2024-08-18T13:41:49.543 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/38 file after the snapshot 2024-08-18T13:41:49.546 INFO:tasks.workunit.client.0.smithi118.stdout: create 279/39 file after the snapshot 2024-08-18T13:41:49.549 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/.snap/snap-subdir-test 2024-08-18T13:41:49.571 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/30 file after the snapshot 2024-08-18T13:41:49.575 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/31 file after the snapshot 2024-08-18T13:41:49.579 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/32 file after the snapshot 2024-08-18T13:41:49.583 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/33 file after the snapshot 2024-08-18T13:41:49.589 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/34 file after the snapshot 2024-08-18T13:41:49.589 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/35 file after the snapshot 2024-08-18T13:41:49.592 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/36 file after the snapshot 2024-08-18T13:41:49.596 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/37 file after the snapshot 2024-08-18T13:41:49.599 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/38 file after the snapshot 2024-08-18T13:41:49.602 INFO:tasks.workunit.client.0.smithi118.stdout: create 280/39 file after the snapshot 2024-08-18T13:41:49.605 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/.snap/snap-subdir-test 2024-08-18T13:41:49.630 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/30 file after the snapshot 2024-08-18T13:41:49.634 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/31 file after the snapshot 2024-08-18T13:41:49.638 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/32 file after the snapshot 2024-08-18T13:41:49.642 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/33 file after the snapshot 2024-08-18T13:41:49.646 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/34 file after the snapshot 2024-08-18T13:41:49.649 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/35 file after the snapshot 2024-08-18T13:41:49.652 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/36 file after the snapshot 2024-08-18T13:41:49.655 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/37 file after the snapshot 2024-08-18T13:41:49.658 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/38 file after the snapshot 2024-08-18T13:41:49.661 INFO:tasks.workunit.client.0.smithi118.stdout: create 281/39 file after the snapshot 2024-08-18T13:41:49.664 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/.snap/snap-subdir-test 2024-08-18T13:41:49.689 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/30 file after the snapshot 2024-08-18T13:41:49.692 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/31 file after the snapshot 2024-08-18T13:41:49.696 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/32 file after the snapshot 2024-08-18T13:41:49.699 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/33 file after the snapshot 2024-08-18T13:41:49.703 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/34 file after the snapshot 2024-08-18T13:41:49.706 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/35 file after the snapshot 2024-08-18T13:41:49.710 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/36 file after the snapshot 2024-08-18T13:41:49.714 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/37 file after the snapshot 2024-08-18T13:41:49.715 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/38 file after the snapshot 2024-08-18T13:41:49.719 INFO:tasks.workunit.client.0.smithi118.stdout: create 282/39 file after the snapshot 2024-08-18T13:41:49.721 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/.snap/snap-subdir-test 2024-08-18T13:41:49.749 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/30 file after the snapshot 2024-08-18T13:41:49.753 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/31 file after the snapshot 2024-08-18T13:41:49.757 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/32 file after the snapshot 2024-08-18T13:41:49.772 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/33 file after the snapshot 2024-08-18T13:41:49.836 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/34 file after the snapshot 2024-08-18T13:41:49.840 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/35 file after the snapshot 2024-08-18T13:41:49.845 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/36 file after the snapshot 2024-08-18T13:41:49.849 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/37 file after the snapshot 2024-08-18T13:41:49.852 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/38 file after the snapshot 2024-08-18T13:41:49.854 INFO:tasks.workunit.client.0.smithi118.stdout: create 283/39 file after the snapshot 2024-08-18T13:41:49.858 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/.snap/snap-subdir-test 2024-08-18T13:41:49.916 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/30 file after the snapshot 2024-08-18T13:41:49.919 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/31 file after the snapshot 2024-08-18T13:41:49.924 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/32 file after the snapshot 2024-08-18T13:41:49.928 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/33 file after the snapshot 2024-08-18T13:41:49.931 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/34 file after the snapshot 2024-08-18T13:41:49.934 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/35 file after the snapshot 2024-08-18T13:41:49.937 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/36 file after the snapshot 2024-08-18T13:41:49.940 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/37 file after the snapshot 2024-08-18T13:41:49.943 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/38 file after the snapshot 2024-08-18T13:41:49.947 INFO:tasks.workunit.client.0.smithi118.stdout: create 284/39 file after the snapshot 2024-08-18T13:41:49.950 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/.snap/snap-subdir-test 2024-08-18T13:41:49.973 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/30 file after the snapshot 2024-08-18T13:41:49.976 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/31 file after the snapshot 2024-08-18T13:41:49.980 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/32 file after the snapshot 2024-08-18T13:41:49.985 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/33 file after the snapshot 2024-08-18T13:41:49.989 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/34 file after the snapshot 2024-08-18T13:41:49.993 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/35 file after the snapshot 2024-08-18T13:41:49.995 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/36 file after the snapshot 2024-08-18T13:41:49.999 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/37 file after the snapshot 2024-08-18T13:41:50.002 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/38 file after the snapshot 2024-08-18T13:41:50.005 INFO:tasks.workunit.client.0.smithi118.stdout: create 285/39 file after the snapshot 2024-08-18T13:41:50.009 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/.snap/snap-subdir-test 2024-08-18T13:41:50.030 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/30 file after the snapshot 2024-08-18T13:41:50.033 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/31 file after the snapshot 2024-08-18T13:41:50.038 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/32 file after the snapshot 2024-08-18T13:41:50.041 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/33 file after the snapshot 2024-08-18T13:41:50.045 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/34 file after the snapshot 2024-08-18T13:41:50.049 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/35 file after the snapshot 2024-08-18T13:41:50.051 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/36 file after the snapshot 2024-08-18T13:41:50.054 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/37 file after the snapshot 2024-08-18T13:41:50.057 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/38 file after the snapshot 2024-08-18T13:41:50.060 INFO:tasks.workunit.client.0.smithi118.stdout: create 286/39 file after the snapshot 2024-08-18T13:41:50.062 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/.snap/snap-subdir-test 2024-08-18T13:41:50.084 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/30 file after the snapshot 2024-08-18T13:41:50.087 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/31 file after the snapshot 2024-08-18T13:41:50.090 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/32 file after the snapshot 2024-08-18T13:41:50.094 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/33 file after the snapshot 2024-08-18T13:41:50.098 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/34 file after the snapshot 2024-08-18T13:41:50.101 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/35 file after the snapshot 2024-08-18T13:41:50.104 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/36 file after the snapshot 2024-08-18T13:41:50.107 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/37 file after the snapshot 2024-08-18T13:41:50.110 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/38 file after the snapshot 2024-08-18T13:41:50.113 INFO:tasks.workunit.client.0.smithi118.stdout: create 287/39 file after the snapshot 2024-08-18T13:41:50.116 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/.snap/snap-subdir-test 2024-08-18T13:41:50.136 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/30 file after the snapshot 2024-08-18T13:41:50.139 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/31 file after the snapshot 2024-08-18T13:41:50.142 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/32 file after the snapshot 2024-08-18T13:41:50.147 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/33 file after the snapshot 2024-08-18T13:41:50.150 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/34 file after the snapshot 2024-08-18T13:41:50.154 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/35 file after the snapshot 2024-08-18T13:41:50.157 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/36 file after the snapshot 2024-08-18T13:41:50.160 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/37 file after the snapshot 2024-08-18T13:41:50.162 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/38 file after the snapshot 2024-08-18T13:41:50.165 INFO:tasks.workunit.client.0.smithi118.stdout: create 288/39 file after the snapshot 2024-08-18T13:41:50.168 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/.snap/snap-subdir-test 2024-08-18T13:41:50.191 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/30 file after the snapshot 2024-08-18T13:41:50.193 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/31 file after the snapshot 2024-08-18T13:41:50.197 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/32 file after the snapshot 2024-08-18T13:41:50.201 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/33 file after the snapshot 2024-08-18T13:41:50.205 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/34 file after the snapshot 2024-08-18T13:41:50.207 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/35 file after the snapshot 2024-08-18T13:41:50.210 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/36 file after the snapshot 2024-08-18T13:41:50.213 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/37 file after the snapshot 2024-08-18T13:41:50.215 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/38 file after the snapshot 2024-08-18T13:41:50.218 INFO:tasks.workunit.client.0.smithi118.stdout: create 289/39 file after the snapshot 2024-08-18T13:41:50.221 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/.snap/snap-subdir-test 2024-08-18T13:41:50.249 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/30 file after the snapshot 2024-08-18T13:41:50.251 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/31 file after the snapshot 2024-08-18T13:41:50.255 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/32 file after the snapshot 2024-08-18T13:41:50.260 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/33 file after the snapshot 2024-08-18T13:41:50.263 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/34 file after the snapshot 2024-08-18T13:41:50.266 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/35 file after the snapshot 2024-08-18T13:41:50.269 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/36 file after the snapshot 2024-08-18T13:41:50.272 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/37 file after the snapshot 2024-08-18T13:41:50.276 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/38 file after the snapshot 2024-08-18T13:41:50.278 INFO:tasks.workunit.client.0.smithi118.stdout: create 290/39 file after the snapshot 2024-08-18T13:41:50.281 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/.snap/snap-subdir-test 2024-08-18T13:41:50.305 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/30 file after the snapshot 2024-08-18T13:41:50.308 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/31 file after the snapshot 2024-08-18T13:41:50.312 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/32 file after the snapshot 2024-08-18T13:41:50.316 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/33 file after the snapshot 2024-08-18T13:41:50.319 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/34 file after the snapshot 2024-08-18T13:41:50.326 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/35 file after the snapshot 2024-08-18T13:41:50.327 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/36 file after the snapshot 2024-08-18T13:41:50.329 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/37 file after the snapshot 2024-08-18T13:41:50.332 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/38 file after the snapshot 2024-08-18T13:41:50.334 INFO:tasks.workunit.client.0.smithi118.stdout: create 291/39 file after the snapshot 2024-08-18T13:41:50.336 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/.snap/snap-subdir-test 2024-08-18T13:41:50.360 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/30 file after the snapshot 2024-08-18T13:41:50.363 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/31 file after the snapshot 2024-08-18T13:41:50.367 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/32 file after the snapshot 2024-08-18T13:41:50.372 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/33 file after the snapshot 2024-08-18T13:41:50.375 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/34 file after the snapshot 2024-08-18T13:41:50.378 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/35 file after the snapshot 2024-08-18T13:41:50.382 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/36 file after the snapshot 2024-08-18T13:41:50.385 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/37 file after the snapshot 2024-08-18T13:41:50.387 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/38 file after the snapshot 2024-08-18T13:41:50.390 INFO:tasks.workunit.client.0.smithi118.stdout: create 292/39 file after the snapshot 2024-08-18T13:41:50.394 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/.snap/snap-subdir-test 2024-08-18T13:41:50.415 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/30 file after the snapshot 2024-08-18T13:41:50.417 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/31 file after the snapshot 2024-08-18T13:41:50.420 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/32 file after the snapshot 2024-08-18T13:41:50.424 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/33 file after the snapshot 2024-08-18T13:41:50.427 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/34 file after the snapshot 2024-08-18T13:41:50.430 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/35 file after the snapshot 2024-08-18T13:41:50.433 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/36 file after the snapshot 2024-08-18T13:41:50.437 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/37 file after the snapshot 2024-08-18T13:41:50.440 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/38 file after the snapshot 2024-08-18T13:41:50.442 INFO:tasks.workunit.client.0.smithi118.stdout: create 293/39 file after the snapshot 2024-08-18T13:41:50.446 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/.snap/snap-subdir-test 2024-08-18T13:41:50.474 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/30 file after the snapshot 2024-08-18T13:41:50.476 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/31 file after the snapshot 2024-08-18T13:41:50.479 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/32 file after the snapshot 2024-08-18T13:41:50.483 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/33 file after the snapshot 2024-08-18T13:41:50.486 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/34 file after the snapshot 2024-08-18T13:41:50.490 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/35 file after the snapshot 2024-08-18T13:41:50.493 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/36 file after the snapshot 2024-08-18T13:41:50.497 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/37 file after the snapshot 2024-08-18T13:41:50.500 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/38 file after the snapshot 2024-08-18T13:41:50.502 INFO:tasks.workunit.client.0.smithi118.stdout: create 294/39 file after the snapshot 2024-08-18T13:41:50.505 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/.snap/snap-subdir-test 2024-08-18T13:41:50.531 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/30 file after the snapshot 2024-08-18T13:41:50.531 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/31 file after the snapshot 2024-08-18T13:41:50.534 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/32 file after the snapshot 2024-08-18T13:41:50.537 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/33 file after the snapshot 2024-08-18T13:41:50.540 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/34 file after the snapshot 2024-08-18T13:41:50.544 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/35 file after the snapshot 2024-08-18T13:41:50.548 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/36 file after the snapshot 2024-08-18T13:41:50.551 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/37 file after the snapshot 2024-08-18T13:41:50.554 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/38 file after the snapshot 2024-08-18T13:41:50.556 INFO:tasks.workunit.client.0.smithi118.stdout: create 295/39 file after the snapshot 2024-08-18T13:41:50.559 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/.snap/snap-subdir-test 2024-08-18T13:41:50.581 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/30 file after the snapshot 2024-08-18T13:41:50.585 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/31 file after the snapshot 2024-08-18T13:41:50.589 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/32 file after the snapshot 2024-08-18T13:41:50.593 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/33 file after the snapshot 2024-08-18T13:41:50.597 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/34 file after the snapshot 2024-08-18T13:41:50.600 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/35 file after the snapshot 2024-08-18T13:41:50.603 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/36 file after the snapshot 2024-08-18T13:41:50.606 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/37 file after the snapshot 2024-08-18T13:41:50.609 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/38 file after the snapshot 2024-08-18T13:41:50.612 INFO:tasks.workunit.client.0.smithi118.stdout: create 296/39 file after the snapshot 2024-08-18T13:41:50.615 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/.snap/snap-subdir-test 2024-08-18T13:41:50.638 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/30 file after the snapshot 2024-08-18T13:41:50.642 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/31 file after the snapshot 2024-08-18T13:41:50.647 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/32 file after the snapshot 2024-08-18T13:41:50.651 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/33 file after the snapshot 2024-08-18T13:41:50.655 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/34 file after the snapshot 2024-08-18T13:41:50.658 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/35 file after the snapshot 2024-08-18T13:41:50.661 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/36 file after the snapshot 2024-08-18T13:41:50.664 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/37 file after the snapshot 2024-08-18T13:41:50.667 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/38 file after the snapshot 2024-08-18T13:41:50.670 INFO:tasks.workunit.client.0.smithi118.stdout: create 297/39 file after the snapshot 2024-08-18T13:41:50.672 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/.snap/snap-subdir-test 2024-08-18T13:41:50.698 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/30 file after the snapshot 2024-08-18T13:41:50.700 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/31 file after the snapshot 2024-08-18T13:41:50.704 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/32 file after the snapshot 2024-08-18T13:41:50.709 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/33 file after the snapshot 2024-08-18T13:41:50.713 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/34 file after the snapshot 2024-08-18T13:41:50.717 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/35 file after the snapshot 2024-08-18T13:41:50.720 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/36 file after the snapshot 2024-08-18T13:41:50.723 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/37 file after the snapshot 2024-08-18T13:41:50.727 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/38 file after the snapshot 2024-08-18T13:41:50.730 INFO:tasks.workunit.client.0.smithi118.stdout: create 298/39 file after the snapshot 2024-08-18T13:41:50.734 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/.snap/snap-subdir-test 2024-08-18T13:41:50.756 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/30 file after the snapshot 2024-08-18T13:41:50.759 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/31 file after the snapshot 2024-08-18T13:41:50.763 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/32 file after the snapshot 2024-08-18T13:41:50.766 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/33 file after the snapshot 2024-08-18T13:41:50.770 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/34 file after the snapshot 2024-08-18T13:41:50.786 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/35 file after the snapshot 2024-08-18T13:41:50.833 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/36 file after the snapshot 2024-08-18T13:41:50.837 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/37 file after the snapshot 2024-08-18T13:41:50.841 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/38 file after the snapshot 2024-08-18T13:41:50.845 INFO:tasks.workunit.client.0.smithi118.stdout: create 299/39 file after the snapshot 2024-08-18T13:41:50.847 INFO:tasks.workunit.client.0.smithi118.stdout:Take second snapshot .snap/test2 2024-08-18T13:41:50.986 INFO:tasks.workunit.client.0.smithi118.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2024-08-18T13:41:52.781 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2024-08-18T13:41:52.782 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2024-08-18T13:41:52.783 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2024-08-18T13:41:52.784 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2024-08-18T13:41:52.785 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2024-08-18T13:41:52.786 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2024-08-18T13:41:52.787 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2024-08-18T13:41:54.324 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2024-08-18T13:41:54.325 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2024-08-18T13:41:54.326 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2024-08-18T13:41:54.327 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2024-08-18T13:41:54.328 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2024-08-18T13:41:54.329 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2024-08-18T13:41:54.330 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2024-08-18T13:41:54.331 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2024-08-18T13:41:54.331 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2024-08-18T13:41:55.805 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2024-08-18T13:41:55.806 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2024-08-18T13:41:55.807 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2024-08-18T13:41:55.808 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2024-08-18T13:41:55.809 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2024-08-18T13:41:55.810 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2024-08-18T13:41:55.811 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2024-08-18T13:41:55.812 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2024-08-18T13:41:55.812 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2024-08-18T13:41:57.359 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2024-08-18T13:41:57.360 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2024-08-18T13:41:57.361 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2024-08-18T13:41:57.362 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2024-08-18T13:41:57.363 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2024-08-18T13:41:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2024-08-18T13:41:57.365 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2024-08-18T13:41:57.365 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2024-08-18T13:41:57.365 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2024-08-18T13:41:58.728 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2024-08-18T13:41:58.729 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2024-08-18T13:41:58.730 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2024-08-18T13:41:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2024-08-18T13:41:58.732 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2024-08-18T13:41:58.733 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2024-08-18T13:41:58.734 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2024-08-18T13:41:58.735 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2024-08-18T13:41:58.736 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2024-08-18T13:42:00.193 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2024-08-18T13:42:00.193 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2024-08-18T13:42:00.193 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2024-08-18T13:42:00.193 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2024-08-18T13:42:00.193 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2024-08-18T13:42:00.193 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2024-08-18T13:42:00.193 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2024-08-18T13:42:00.194 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2024-08-18T13:42:00.195 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2024-08-18T13:42:00.196 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2024-08-18T13:42:00.197 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2024-08-18T13:42:00.198 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2024-08-18T13:42:00.199 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2024-08-18T13:42:00.199 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2024-08-18T13:42:00.199 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2024-08-18T13:42:00.199 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2024-08-18T13:42:01.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2024-08-18T13:42:01.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2024-08-18T13:42:01.589 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2024-08-18T13:42:01.590 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2024-08-18T13:42:01.591 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2024-08-18T13:42:01.592 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2024-08-18T13:42:01.593 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2024-08-18T13:42:01.594 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2024-08-18T13:42:01.594 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2024-08-18T13:42:01.594 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2024-08-18T13:42:01.594 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2024-08-18T13:42:01.594 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2024-08-18T13:42:01.594 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2024-08-18T13:42:01.594 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2024-08-18T13:42:01.671 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862e7f4700 1 Processor -- start 2024-08-18T13:42:01.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862e7f4700 1 -- start start 2024-08-18T13:42:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862e7f4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f86280aebc0 0x7f86280b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862e7f4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8628155330 con 0x7f86280aef90 2024-08-18T13:42:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862e7f4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86281555d0 con 0x7f86280c6880 2024-08-18T13:42:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862e7f4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8628155140 con 0x7f86280aebc0 2024-08-18T13:42:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862cff1700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f86280aef90 0x7f86280abad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41112/0 (socket says 172.21.15.118:41112) 2024-08-18T13:42:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862cff1700 1 -- 172.21.15.118:0/3921609569 learned_addr learned my addr 172.21.15.118:0/3921609569 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:01.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f862d7f2700 1 --2- 172.21.15.118:0/3921609569 >> v2:172.21.15.118:3300/0 conn(0x7f86280aebc0 0x7f86280b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f861ffff700 1 -- 172.21.15.118:0/3921609569 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398405848 0 0) 0x7f8628155330 con 0x7f86280aef90 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.784+0000 7f861ffff700 1 -- 172.21.15.118:0/3921609569 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8614003570 con 0x7f86280aef90 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862d7f2700 1 -- 172.21.15.118:0/3921609569 >> v1:172.21.15.178:6789/0 conn(0x7f86280c6880 legacy=0x7f86280c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862d7f2700 1 -- 172.21.15.118:0/3921609569 >> v1:172.21.15.118:6789/0 conn(0x7f86280aef90 legacy=0x7f86280abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862d7f2700 1 -- 172.21.15.118:0/3921609569 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628155870 con 0x7f86280aebc0 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862d7f2700 1 --2- 172.21.15.118:0/3921609569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280b1640 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8628155a60 tx=0x7f86240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d9d12abb008615 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f861ffff700 1 -- 172.21.15.118:0/3921609569 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624011630 con 0x7f86280aebc0 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f861ffff700 1 -- 172.21.15.118:0/3921609569 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8624011790 con 0x7f86280aebc0 2024-08-18T13:42:01.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f861ffff700 1 -- 172.21.15.118:0/3921609569 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624011b30 con 0x7f86280aebc0 2024-08-18T13:42:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862e7f4700 1 -- 172.21.15.118:0/3921609569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 msgr2=0x7f86280b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862e7f4700 1 --2- 172.21.15.118:0/3921609569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280b1640 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8628155a60 tx=0x7f86240091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862e7f4700 1 -- 172.21.15.118:0/3921609569 shutdown_connections 2024-08-18T13:42:01.791 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862e7f4700 1 --2- 172.21.15.118:0/3921609569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280b1640 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862e7f4700 1 -- 172.21.15.118:0/3921609569 >> 172.21.15.118:0/3921609569 conn(0x7f86280cbba0 msgr2=0x7f86280cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862e7f4700 1 -- 172.21.15.118:0/3921609569 shutdown_connections 2024-08-18T13:42:01.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.788+0000 7f862e7f4700 1 -- 172.21.15.118:0/3921609569 wait complete. 2024-08-18T13:42:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 Processor -- start 2024-08-18T13:42:01.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 -- start start 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86280aef90 0x7f8628151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86280c6880 0x7f862814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862814e9b0 con 0x7f86280aebc0 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c2e60 con 0x7f86280aef90 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862806a070 con 0x7f86280c6880 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:55506/0 (socket says 172.21.15.118:55506) 2024-08-18T13:42:01.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 -- 172.21.15.118:0/1755965669 learned_addr learned my addr 172.21.15.118:0/1755965669 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862cff1700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86280aef90 0x7f8628151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862dff3700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86280c6880 0x7f862814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 -- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86280c6880 msgr2=0x7f862814d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86280c6880 0x7f862814d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 -- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86280aef90 msgr2=0x7f8628151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86280aef90 0x7f8628151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 -- 172.21.15.118:0/1755965669 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8624009420 con 0x7f86280aebc0 2024-08-18T13:42:01.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862d7f2700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280c32f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86280b0b70 tx=0x7f8624016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f861d7fa700 1 -- 172.21.15.118:0/1755965669 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862401cc00 con 0x7f86280aebc0 2024-08-18T13:42:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f861d7fa700 1 -- 172.21.15.118:0/1755965669 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862401cd80 con 0x7f86280aebc0 2024-08-18T13:42:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8628156b10 con 0x7f86280aebc0 2024-08-18T13:42:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f861d7fa700 1 -- 172.21.15.118:0/1755965669 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862401a6a0 con 0x7f86280aebc0 2024-08-18T13:42:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8628156db0 con 0x7f86280aebc0 2024-08-18T13:42:01.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.792+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86280713a0 con 0x7f86280aebc0 2024-08-18T13:42:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.796+0000 7f861d7fa700 1 -- 172.21.15.118:0/1755965669 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f862401aa90 con 0x7f86280aebc0 2024-08-18T13:42:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.796+0000 7f861d7fa700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f860c060860 0x7f860c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.796+0000 7f861d7fa700 1 -- 172.21.15.118:0/1755965669 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6715+0+0 (secure 0 0 0) 0x7f862407e9a0 con 0x7f86280aebc0 2024-08-18T13:42:01.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.796+0000 7f862cff1700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f860c060860 0x7f860c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:01.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.796+0000 7f862cff1700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f860c060860 0x7f860c062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86280c28d0 tx=0x7f8618009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:01.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:01.800+0000 7f861d7fa700 1 -- 172.21.15.118:0/1755965669 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f862404ea00 con 0x7f86280aebc0 2024-08-18T13:42:02.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.020+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86280c8b40 con 0x7f86280aebc0 2024-08-18T13:42:02.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f861d7fa700 1 -- 172.21.15.118:0/1755965669 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7f8624052020 con 0x7f86280aebc0 2024-08-18T13:42:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:02.026 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:02.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f860c060860 msgr2=0x7f860c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f860c060860 0x7f860c062d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86280c28d0 tx=0x7f8618009040 comp rx=0 tx=0).stop 2024-08-18T13:42:02.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 msgr2=0x7f86280c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280c32f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f86280b0b70 tx=0x7f8624016000 comp rx=0 tx=0).stop 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 shutdown_connections 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f860c060860 0x7f860c062d10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86280c6880 0x7f862814d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86280aef90 0x7f8628151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 --2- 172.21.15.118:0/1755965669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86280aebc0 0x7f86280c32f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 >> 172.21.15.118:0/1755965669 conn(0x7f86280cbba0 msgr2=0x7f8628152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 shutdown_connections 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.024+0000 7f862e7f4700 1 -- 172.21.15.118:0/1755965669 wait complete. 2024-08-18T13:42:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:42:02.040 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=0) 2024-08-18T13:42:02.040 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-18T13:42:02.040 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:42:02.059 INFO:tasks.ceph.mds.d:Stopped 2024-08-18T13:42:02.059 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:02.168 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.164+0000 7f801359e700 1 Processor -- start 2024-08-18T13:42:02.168 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.164+0000 7f801359e700 1 -- start start 2024-08-18T13:42:02.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f801414b840 0x7f8014155580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:02.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f801406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:02.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140c2de0 con 0x7f801406b180 2024-08-18T13:42:02.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80140c1d80 con 0x7f801414b840 2024-08-18T13:42:02.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80140c30c0 con 0x7f8014069fc0 2024-08-18T13:42:02.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f801414b840 0x7f8014155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:02.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f801414b840 0x7f8014155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:55510/0 (socket says 172.21.15.118:55510) 2024-08-18T13:42:02.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 -- 172.21.15.118:0/1369120456 learned_addr learned my addr 172.21.15.118:0/1369120456 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:02.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f8009d9b700 1 --2- 172.21.15.118:0/1369120456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f801406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:02.172 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 -- 172.21.15.118:0/1369120456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 msgr2=0x7f801406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 --2- 172.21.15.118:0/1369120456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f801406bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 -- 172.21.15.118:0/1369120456 >> v1:172.21.15.178:6789/0 conn(0x7f801406b180 legacy=0x7f80140c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 -- 172.21.15.118:0/1369120456 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801406a490 con 0x7f801414b840 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801259c700 1 --2- 172.21.15.118:0/1369120456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801414b840 0x7f8014155580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f800400ab50 tx=0x7f800400a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6599463ccd77d44 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f8011d9b700 1 -- 172.21.15.118:0/1369120456 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400ca40 con 0x7f801414b840 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f8011d9b700 1 -- 172.21.15.118:0/1369120456 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800400cba0 con 0x7f801414b840 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f8011d9b700 1 -- 172.21.15.118:0/1369120456 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004018400 con 0x7f801414b840 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- 172.21.15.118:0/1369120456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801414b840 msgr2=0x7f8014155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 --2- 172.21.15.118:0/1369120456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801414b840 0x7f8014155580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f800400ab50 tx=0x7f800400a3f0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- 172.21.15.118:0/1369120456 shutdown_connections 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 --2- 172.21.15.118:0/1369120456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f801406bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 --2- 172.21.15.118:0/1369120456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801414b840 0x7f8014155580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- 172.21.15.118:0/1369120456 >> 172.21.15.118:0/1369120456 conn(0x7f80140c81e0 msgr2=0x7f80140c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:02.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- 172.21.15.118:0/1369120456 shutdown_connections 2024-08-18T13:42:02.174 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.168+0000 7f801359e700 1 -- 172.21.15.118:0/1369120456 wait complete. 2024-08-18T13:42:02.174 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 Processor -- start 2024-08-18T13:42:02.174 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 -- start start 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f80141518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801406b180 0x7f8014147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f801414b840 0x7f80140be570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014146e10 con 0x7f801406b180 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801414ea60 con 0x7f801414b840 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8014164190 con 0x7f8014069fc0 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801406b180 0x7f8014147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:02.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801406b180 0x7f8014147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:55518/0 (socket says 172.21.15.118:55518) 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 -- 172.21.15.118:0/1602149254 learned_addr learned my addr 172.21.15.118:0/1602149254 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f801259c700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f80141518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8012d9d700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f801414b840 0x7f80140be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 -- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 msgr2=0x7f80141518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f80141518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 -- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f801414b840 msgr2=0x7f80140be570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f801414b840 0x7f80140be570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 -- 172.21.15.118:0/1602149254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800400a690 con 0x7f801406b180 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.172+0000 7f8009d9b700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801406b180 0x7f8014147230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00ec80 tx=0x7f7ffc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f800affd700 1 -- 172.21.15.118:0/1602149254 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc00c8e0 con 0x7f801406b180 2024-08-18T13:42:02.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f800affd700 1 -- 172.21.15.118:0/1602149254 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc00ca40 con 0x7f801406b180 2024-08-18T13:42:02.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80141637b0 con 0x7f801406b180 2024-08-18T13:42:02.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f800affd700 1 -- 172.21.15.118:0/1602149254 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc00ce00 con 0x7f801406b180 2024-08-18T13:42:02.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80141639e0 con 0x7f801406b180 2024-08-18T13:42:02.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8014163c00 con 0x7f801406b180 2024-08-18T13:42:02.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f800affd700 1 -- 172.21.15.118:0/1602149254 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7ffc013680 con 0x7f801406b180 2024-08-18T13:42:02.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f800affd700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7ff4060860 0x7f7ff4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:02.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f800affd700 1 -- 172.21.15.118:0/1602149254 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6715+0+0 (secure 0 0 0) 0x7f7ffc07d270 con 0x7f801406b180 2024-08-18T13:42:02.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f801259c700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7ff4060860 0x7f7ff4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:02.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.176+0000 7f801259c700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7ff4060860 0x7f7ff4062d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f800400ab50 tx=0x7f800401a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:02.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.184+0000 7f800affd700 1 -- 172.21.15.118:0/1602149254 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7ffc04d2d0 con 0x7f801406b180 2024-08-18T13:42:02.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.384+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80140713a0 con 0x7f801406b180 2024-08-18T13:42:02.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.384+0000 7f800affd700 1 -- 172.21.15.118:0/1602149254 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7f7ffc0508f0 con 0x7f801406b180 2024-08-18T13:42:02.388 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:02.388 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:02.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7ff4060860 msgr2=0x7f7ff4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7ff4060860 0x7f7ff4062d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f800400ab50 tx=0x7f800401a000 comp rx=0 tx=0).stop 2024-08-18T13:42:02.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801406b180 msgr2=0x7f8014147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801406b180 0x7f8014147230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00ec80 tx=0x7f7ffc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 shutdown_connections 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7ff4060860 0x7f7ff4062d10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f801414b840 0x7f80140be570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f801406b180 0x7f8014147230 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 --2- 172.21.15.118:0/1602149254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8014069fc0 0x7f80141518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 >> 172.21.15.118:0/1602149254 conn(0x7f80140c81e0 msgr2=0x7f801405e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 shutdown_connections 2024-08-18T13:42:02.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:02.388+0000 7f801359e700 1 -- 172.21.15.118:0/1602149254 wait complete. 2024-08-18T13:42:02.392 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:42:02.402 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:42:04.402 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:04.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcb523700 1 Processor -- start 2024-08-18T13:42:04.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcb523700 1 -- start start 2024-08-18T13:42:04.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcb523700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aebc0 0x7f8fc40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:04.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcb523700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8fc40aef90 0x7f8fc40abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:04.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcb523700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc4155330 con 0x7f8fc40c6880 2024-08-18T13:42:04.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcb523700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc4155140 con 0x7f8fc40aef90 2024-08-18T13:42:04.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcb523700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40af660 con 0x7f8fc40aebc0 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fc9d20700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8fc40aef90 0x7f8fc40abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcad22700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f8fc40c6880 0x7f8fc40c6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:50988/0 (socket says 172.21.15.118:50988) 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fcad22700 1 -- 172.21.15.118:0/1592904503 learned_addr learned my addr 172.21.15.118:0/1592904503 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fca521700 1 --2- 172.21.15.118:0/1592904503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aebc0 0x7f8fc40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fca521700 1 -- 172.21.15.118:0/1592904503 >> v2:172.21.15.118:3300/0 conn(0x7f8fc40aef90 msgr2=0x7f8fc40abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fca521700 1 --2- 172.21.15.118:0/1592904503 >> v2:172.21.15.118:3300/0 conn(0x7f8fc40aef90 0x7f8fc40abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fca521700 1 -- 172.21.15.118:0/1592904503 >> v1:172.21.15.118:6789/0 conn(0x7f8fc40c6880 legacy=0x7f8fc40c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fca521700 1 -- 172.21.15.118:0/1592904503 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc41555d0 con 0x7f8fc40aebc0 2024-08-18T13:42:04.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fc9d20700 1 --2- 172.21.15.118:0/1592904503 >> v2:172.21.15.118:3300/0 conn(0x7f8fc40aef90 0x7f8fc40abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:42:04.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fca521700 1 --2- 172.21.15.118:0/1592904503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aebc0 0x7f8fc40b1640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00098e0 tx=0x7f8fc00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=164f22dbf1b70021 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:04.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fc951f700 1 -- 172.21.15.118:0/1592904503 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000b640 con 0x7f8fc40aebc0 2024-08-18T13:42:04.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fc951f700 1 -- 172.21.15.118:0/1592904503 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc000b7a0 con 0x7f8fc40aebc0 2024-08-18T13:42:04.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.512+0000 7f8fc951f700 1 -- 172.21.15.118:0/1592904503 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000bb40 con 0x7f8fc40aebc0 2024-08-18T13:42:04.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 -- 172.21.15.118:0/1592904503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aebc0 msgr2=0x7f8fc40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:04.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 --2- 172.21.15.118:0/1592904503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aebc0 0x7f8fc40b1640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00098e0 tx=0x7f8fc00091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 -- 172.21.15.118:0/1592904503 shutdown_connections 2024-08-18T13:42:04.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 --2- 172.21.15.118:0/1592904503 >> v2:172.21.15.118:3300/0 conn(0x7f8fc40aef90 0x7f8fc40abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 --2- 172.21.15.118:0/1592904503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aebc0 0x7f8fc40b1640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 -- 172.21.15.118:0/1592904503 >> 172.21.15.118:0/1592904503 conn(0x7f8fc40cbba0 msgr2=0x7f8fc40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:04.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 -- 172.21.15.118:0/1592904503 shutdown_connections 2024-08-18T13:42:04.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.516+0000 7f8fcb523700 1 -- 172.21.15.118:0/1592904503 wait complete. 2024-08-18T13:42:04.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 Processor -- start 2024-08-18T13:42:04.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 -- start start 2024-08-18T13:42:04.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc40aebc0 0x7f8fc40c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:04.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aef90 0x7f8fc414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:04.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc40c6880 0x7f8fc414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4169720 con 0x7f8fc40c6880 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4152710 con 0x7f8fc40aebc0 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c3c30 con 0x7f8fc40aef90 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc40c6880 0x7f8fc414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc40c6880 0x7f8fc414def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41578/0 (socket says 172.21.15.118:41578) 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 -- 172.21.15.118:0/4136047105 learned_addr learned my addr 172.21.15.118:0/4136047105 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fc9d20700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aef90 0x7f8fc414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:04.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fca521700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc40aebc0 0x7f8fc40c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 -- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aef90 msgr2=0x7f8fc414fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aef90 0x7f8fc414fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 -- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc40aebc0 msgr2=0x7f8fc40c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc40aebc0 0x7f8fc40c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 -- 172.21.15.118:0/4136047105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb8009420 con 0x7f8fc40c6880 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcad22700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc40c6880 0x7f8fc414def0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00cbc0 tx=0x7f8fbc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fb67fc700 1 -- 172.21.15.118:0/4136047105 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00a7e0 con 0x7f8fc40c6880 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fb67fc700 1 -- 172.21.15.118:0/4136047105 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc00a940 con 0x7f8fc40c6880 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc0009420 con 0x7f8fc40c6880 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.520+0000 7f8fb67fc700 1 -- 172.21.15.118:0/4136047105 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00acc0 con 0x7f8fc40c6880 2024-08-18T13:42:04.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.524+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40c9c30 con 0x7f8fc40c6880 2024-08-18T13:42:04.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.524+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40c4660 con 0x7f8fc40c6880 2024-08-18T13:42:04.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.524+0000 7f8fb67fc700 1 -- 172.21.15.118:0/4136047105 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8fbc007300 con 0x7f8fc40c6880 2024-08-18T13:42:04.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.524+0000 7f8fb67fc700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8fa8060860 0x7f8fa8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:04.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.524+0000 7f8fb67fc700 1 -- 172.21.15.118:0/4136047105 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6715+0+0 (secure 0 0 0) 0x7f8fbc07d270 con 0x7f8fc40c6880 2024-08-18T13:42:04.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.524+0000 7f8fca521700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8fa8060860 0x7f8fa8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:04.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.528+0000 7f8fca521700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8fa8060860 0x7f8fa8062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40c2a30 tx=0x7f8fc0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:04.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.536+0000 7f8fb67fc700 1 -- 172.21.15.118:0/4136047105 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8fc40c4660 con 0x7f8fc40c6880 2024-08-18T13:42:04.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.732+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc40c4660 con 0x7f8fc40c6880 2024-08-18T13:42:04.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fb67fc700 1 -- 172.21.15.118:0/4136047105 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7f8fc40c4660 con 0x7f8fc40c6880 2024-08-18T13:42:04.738 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:04.738 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:04.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8fa8060860 msgr2=0x7f8fa8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8fa8060860 0x7f8fa8062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40c2a30 tx=0x7f8fc0017000 comp rx=0 tx=0).stop 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc40c6880 msgr2=0x7f8fc414def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc40c6880 0x7f8fc414def0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00cbc0 tx=0x7f8fbc0019c0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 shutdown_connections 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8fa8060860 0x7f8fa8062d10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fc40c6880 0x7f8fc414def0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8fc40aef90 0x7f8fc414fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 --2- 172.21.15.118:0/4136047105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fc40aebc0 0x7f8fc40c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 >> 172.21.15.118:0/4136047105 conn(0x7f8fc40cbba0 msgr2=0x7f8fc41547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 shutdown_connections 2024-08-18T13:42:04.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:04.736+0000 7f8fcb523700 1 -- 172.21.15.118:0/4136047105 wait complete. 2024-08-18T13:42:04.742 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:42:04.752 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:42:06.754 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:06.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeffff700 1 Processor -- start 2024-08-18T13:42:06.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeffff700 1 -- start start 2024-08-18T13:42:06.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf0155580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:06.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00c2de0 con 0x7fbbf014b840 2024-08-18T13:42:06.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeffff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00c30c0 con 0x7fbbf006b180 2024-08-18T13:42:06.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00c1d80 con 0x7fbbf0069fc0 2024-08-18T13:42:06.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeeffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbbf014b840 0x7fbbf006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:50998/0 (socket says 172.21.15.118:50998) 2024-08-18T13:42:06.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbeeffd700 1 -- 172.21.15.118:0/1904982127 learned_addr learned my addr 172.21.15.118:0/1904982127 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:06.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbee7fc700 1 --2- 172.21.15.118:0/1904982127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf0155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:06.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741039923 0 0) 0x7fbbf00c2de0 con 0x7fbbf014b840 2024-08-18T13:42:06.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd8003570 con 0x7fbbf014b840 2024-08-18T13:42:06.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935138628 0 0) 0x7fbbf00c30c0 con 0x7fbbf006b180 2024-08-18T13:42:06.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf00c2de0 con 0x7fbbf006b180 2024-08-18T13:42:06.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699224756 0 0) 0x7fbbd8003570 con 0x7fbbf014b840 2024-08-18T13:42:06.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbee7fc700 1 -- 172.21.15.118:0/1904982127 >> v1:172.21.15.178:6789/0 conn(0x7fbbf006b180 legacy=0x7fbbf00c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:06.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.871+0000 7fbbee7fc700 1 -- 172.21.15.118:0/1904982127 >> v1:172.21.15.118:6789/0 conn(0x7fbbf014b840 legacy=0x7fbbf006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:06.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbee7fc700 1 -- 172.21.15.118:0/1904982127 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf006a490 con 0x7fbbf0069fc0 2024-08-18T13:42:06.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbee7fc700 1 --2- 172.21.15.118:0/1904982127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf0155580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009910 tx=0x7fbbe40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb36a3559397981 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:06.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0001470 con 0x7fbbf0069fc0 2024-08-18T13:42:06.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4011420 con 0x7fbbf0069fc0 2024-08-18T13:42:06.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbedffb700 1 -- 172.21.15.118:0/1904982127 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40117a0 con 0x7fbbf0069fc0 2024-08-18T13:42:06.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbeffff700 1 -- 172.21.15.118:0/1904982127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 msgr2=0x7fbbf0155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:06.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1904982127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf0155580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009910 tx=0x7fbbe40091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:06.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbeffff700 1 -- 172.21.15.118:0/1904982127 shutdown_connections 2024-08-18T13:42:06.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1904982127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf0155580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:06.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbeffff700 1 -- 172.21.15.118:0/1904982127 >> 172.21.15.118:0/1904982127 conn(0x7fbbf00c81e0 msgr2=0x7fbbf00c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:06.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbeffff700 1 -- 172.21.15.118:0/1904982127 shutdown_connections 2024-08-18T13:42:06.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.875+0000 7fbbeffff700 1 -- 172.21.15.118:0/1904982127 wait complete. 2024-08-18T13:42:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 Processor -- start 2024-08-18T13:42:06.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 -- start start 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf006b180 0x7fbbf014e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbf014b840 0x7fbbf00be320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0154c90 con 0x7fbbf006b180 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0165320 con 0x7fbbf014b840 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0165600 con 0x7fbbf0069fc0 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf006b180 0x7fbbf014e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf006b180 0x7fbbf014e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41594/0 (socket says 172.21.15.118:41594) 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 -- 172.21.15.118:0/1199106459 learned_addr learned my addr 172.21.15.118:0/1199106459 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbeeffd700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:06.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbef7fe700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbf014b840 0x7fbbf00be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:06.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 -- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 msgr2=0x7fbbf014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:06.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:06.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 -- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbf014b840 msgr2=0x7fbbf00be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:06.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbf014b840 0x7fbbf00be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:06.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.879+0000 7fbbee7fc700 1 -- 172.21.15.118:0/1199106459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe4009420 con 0x7fbbf006b180 2024-08-18T13:42:06.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbee7fc700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf006b180 0x7fbbf014e8d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009860 tx=0x7fbbe4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:06.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbd6ffd700 1 -- 172.21.15.118:0/1199106459 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401caf0 con 0x7fbbf006b180 2024-08-18T13:42:06.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbd6ffd700 1 -- 172.21.15.118:0/1199106459 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe401cc50 con 0x7fbbf006b180 2024-08-18T13:42:06.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf01658e0 con 0x7fbbf006b180 2024-08-18T13:42:06.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbd6ffd700 1 -- 172.21.15.118:0/1199106459 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401a450 con 0x7fbbf006b180 2024-08-18T13:42:06.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf0152270 con 0x7fbbf006b180 2024-08-18T13:42:06.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbd6ffd700 1 -- 172.21.15.118:0/1199106459 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbbe401a8a0 con 0x7fbbf006b180 2024-08-18T13:42:06.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf0165160 con 0x7fbbf006b180 2024-08-18T13:42:06.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbd6ffd700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbdc060890 0x7fbbdc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:06.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbd6ffd700 1 -- 172.21.15.118:0/1199106459 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6715+0+0 (secure 0 0 0) 0x7fbbe407e7d0 con 0x7fbbf006b180 2024-08-18T13:42:06.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbeeffd700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbdc060890 0x7fbbdc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:06.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.883+0000 7fbbeeffd700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbdc060890 0x7fbbdc062d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00011e0 tx=0x7fbbe0000bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:06.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:06.891+0000 7fbbd6ffd700 1 -- 172.21.15.118:0/1199106459 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbbe404e830 con 0x7fbbf006b180 2024-08-18T13:42:07.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.095+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbf0165af0 con 0x7fbbf006b180 2024-08-18T13:42:07.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.095+0000 7fbbd6ffd700 1 -- 172.21.15.118:0/1199106459 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7fbbe4051e50 con 0x7fbbf006b180 2024-08-18T13:42:07.100 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:07.100 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:07.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbdc060890 msgr2=0x7fbbdc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:07.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbdc060890 0x7fbbdc062d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00011e0 tx=0x7fbbe0000bc0 comp rx=0 tx=0).stop 2024-08-18T13:42:07.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf006b180 msgr2=0x7fbbf014e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf006b180 0x7fbbf014e8d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009860 tx=0x7fbbe4016000 comp rx=0 tx=0).stop 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 shutdown_connections 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbbdc060890 0x7fbbdc062d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbbf014b840 0x7fbbf00be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf006b180 0x7fbbf014e8d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 --2- 172.21.15.118:0/1199106459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbbf0069fc0 0x7fbbf014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 >> 172.21.15.118:0/1199106459 conn(0x7fbbf00c81e0 msgr2=0x7fbbf006bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 shutdown_connections 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:07.099+0000 7fbbeffff700 1 -- 172.21.15.118:0/1199106459 wait complete. 2024-08-18T13:42:07.103 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:42:07.118 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:42:09.119 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:09.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 Processor -- start 2024-08-18T13:42:09.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- start start 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6ac0be340 0x7fa6ac0be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa6ac15a310 0x7fa6ac06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac0c3020 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac0c38c0 con 0x7fa6ac0be340 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac06e3b0 con 0x7fa6ac15a310 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6ac0be340 0x7fa6ac0be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6ac0be340 0x7fa6ac0be710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41602/0 (socket says 172.21.15.118:41602) 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 -- 172.21.15.118:0/3998229516 learned_addr learned my addr 172.21.15.118:0/3998229516 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b0e83700 1 --2- 172.21.15.118:0/3998229516 >> v2:172.21.15.178:3300/0 conn(0x7fa6ac15a310 0x7fa6ac06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 -- 172.21.15.118:0/3998229516 >> v2:172.21.15.178:3300/0 conn(0x7fa6ac15a310 msgr2=0x7fa6ac06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 --2- 172.21.15.118:0/3998229516 >> v2:172.21.15.178:3300/0 conn(0x7fa6ac15a310 0x7fa6ac06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 -- 172.21.15.118:0/3998229516 >> v1:172.21.15.178:6789/0 conn(0x7fa6ac0c3d50 legacy=0x7fa6ac06b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:09.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 -- 172.21.15.118:0/3998229516 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0673d0 con 0x7fa6ac0be340 2024-08-18T13:42:09.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b0e83700 1 --2- 172.21.15.118:0/3998229516 >> v2:172.21.15.178:3300/0 conn(0x7fa6ac15a310 0x7fa6ac06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b1684700 1 --2- 172.21.15.118:0/3998229516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0be340 0x7fa6ac0be710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac068f50 tx=0x7fa6a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f026d8cc4c2f1916 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa69ffff700 1 -- 172.21.15.118:0/3998229516 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800b860 con 0x7fa6ac0be340 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa69ffff700 1 -- 172.21.15.118:0/3998229516 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a800b9c0 con 0x7fa6ac0be340 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa69ffff700 1 -- 172.21.15.118:0/3998229516 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800bd60 con 0x7fa6ac0be340 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- 172.21.15.118:0/3998229516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0be340 msgr2=0x7fa6ac0be710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/3998229516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0be340 0x7fa6ac0be710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac068f50 tx=0x7fa6a8009250 comp rx=0 tx=0).stop 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- 172.21.15.118:0/3998229516 shutdown_connections 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/3998229516 >> v2:172.21.15.178:3300/0 conn(0x7fa6ac15a310 0x7fa6ac06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/3998229516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0be340 0x7fa6ac0be710 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- 172.21.15.118:0/3998229516 >> 172.21.15.118:0/3998229516 conn(0x7fa6ac0c7c40 msgr2=0x7fa6ac0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- 172.21.15.118:0/3998229516 shutdown_connections 2024-08-18T13:42:09.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.235+0000 7fa6b38e8700 1 -- 172.21.15.118:0/3998229516 wait complete. 2024-08-18T13:42:09.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 Processor -- start 2024-08-18T13:42:09.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 -- start start 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6ac0be340 0x7fa6ac15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0c3d50 0x7fa6ac15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ac15a310 0x7fa6ac151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac155ff0 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac15af60 con 0x7fa6ac15a310 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b38e8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac154e80 con 0x7fa6ac0be340 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b0e83700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0c3d50 0x7fa6ac15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:09.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b1684700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6ac0be340 0x7fa6ac15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:09.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b0e83700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0c3d50 0x7fa6ac15b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41614/0 (socket says 172.21.15.118:41614) 2024-08-18T13:42:09.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b0e83700 1 -- 172.21.15.118:0/1292847825 learned_addr learned my addr 172.21.15.118:0/1292847825 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:09.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b1e85700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ac15a310 0x7fa6ac151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.239+0000 7fa6b0e83700 1 -- 172.21.15.118:0/1292847825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6ac0be340 msgr2=0x7fa6ac15b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b0e83700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6ac0be340 0x7fa6ac15b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b0e83700 1 -- 172.21.15.118:0/1292847825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ac15a310 msgr2=0x7fa6ac151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b0e83700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ac15a310 0x7fa6ac151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b0e83700 1 -- 172.21.15.118:0/1292847825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80094f0 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b0e83700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0c3d50 0x7fa6ac15b930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000d860 tx=0x7fa6a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa69d7fa700 1 -- 172.21.15.118:0/1292847825 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00125f0 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa69d7fa700 1 -- 172.21.15.118:0/1292847825 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a0012750 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac155160 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa69d7fa700 1 -- 172.21.15.118:0/1292847825 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0012af0 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac155720 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac15b110 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.243+0000 7fa69d7fa700 1 -- 172.21.15.118:0/1292847825 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa6a0013480 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.247+0000 7fa69d7fa700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa690060890 0x7fa690062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:09.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.247+0000 7fa6b1684700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa690060890 0x7fa690062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:09.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.247+0000 7fa69d7fa700 1 -- 172.21.15.118:0/1292847825 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6715+0+0 (secure 0 0 0) 0x7fa6a007d060 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.247+0000 7fa6b1684700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa690060890 0x7fa690062d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac068f50 tx=0x7fa6a8017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:09.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.251+0000 7fa69d7fa700 1 -- 172.21.15.118:0/1292847825 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa6ac15b110 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.455+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6ac15b0d0 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.455+0000 7fa69d7fa700 1 -- 172.21.15.118:0/1292847825 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7fa6a004d0c0 con 0x7fa6ac0c3d50 2024-08-18T13:42:09.460 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:09.460 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa690060890 msgr2=0x7fa690062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa690060890 0x7fa690062d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac068f50 tx=0x7fa6a8017000 comp rx=0 tx=0).stop 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0c3d50 msgr2=0x7fa6ac15b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0c3d50 0x7fa6ac15b930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000d860 tx=0x7fa6a0008040 comp rx=0 tx=0).stop 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 shutdown_connections 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa690060890 0x7fa690062d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6ac15a310 0x7fa6ac151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6ac0c3d50 0x7fa6ac15b930 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 --2- 172.21.15.118:0/1292847825 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6ac0be340 0x7fa6ac15b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:09.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 >> 172.21.15.118:0/1292847825 conn(0x7fa6ac0c7c40 msgr2=0x7fa6ac0c8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:09.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 shutdown_connections 2024-08-18T13:42:09.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:09.459+0000 7fa6b38e8700 1 -- 172.21.15.118:0/1292847825 wait complete. 2024-08-18T13:42:09.463 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:42:09.477 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:42:11.478 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:11.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe7260a8700 1 Processor -- start 2024-08-18T13:42:11.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe7260a8700 1 -- start start 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe7260a8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe7200c0cf0 0x7fe720069d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe7260a8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe720068260 0x7fe72014f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe7260a8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe720068d50 con 0x7fe7200897f0 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe7260a8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7200c2270 con 0x7fe7200c0cf0 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe7260a8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200698b0 con 0x7fe720068260 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71f7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe7200c0cf0 0x7fe720069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71ffff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fe7200897f0 0x7fe7201588d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:51014/0 (socket says 172.21.15.118:51014) 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71ffff700 1 -- 172.21.15.118:0/1805842707 learned_addr learned my addr 172.21.15.118:0/1805842707 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:11.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71effd700 1 --2- 172.21.15.118:0/1805842707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe720068260 0x7fe72014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:11.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71f7fe700 1 -- 172.21.15.118:0/1805842707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe720068260 msgr2=0x7fe72014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:11.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71f7fe700 1 --2- 172.21.15.118:0/1805842707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe720068260 0x7fe72014f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71f7fe700 1 -- 172.21.15.118:0/1805842707 >> v1:172.21.15.118:6789/0 conn(0x7fe7200897f0 legacy=0x7fe7201588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:11.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71f7fe700 1 -- 172.21.15.118:0/1805842707 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720156090 con 0x7fe7200c0cf0 2024-08-18T13:42:11.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71f7fe700 1 --2- 172.21.15.118:0/1805842707 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7200c0cf0 0x7fe720069d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe7140098e0 tx=0x7fe7140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c15a3e5d126675 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:11.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71e7fc700 1 -- 172.21.15.118:0/1805842707 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71400b640 con 0x7fe7200c0cf0 2024-08-18T13:42:11.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71e7fc700 1 -- 172.21.15.118:0/1805842707 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71400b7a0 con 0x7fe7200c0cf0 2024-08-18T13:42:11.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.595+0000 7fe71e7fc700 1 -- 172.21.15.118:0/1805842707 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71400bb40 con 0x7fe7200c0cf0 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 -- 172.21.15.118:0/1805842707 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7200c0cf0 msgr2=0x7fe720069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 --2- 172.21.15.118:0/1805842707 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7200c0cf0 0x7fe720069d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe7140098e0 tx=0x7fe7140091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 -- 172.21.15.118:0/1805842707 shutdown_connections 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 --2- 172.21.15.118:0/1805842707 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe720068260 0x7fe72014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 --2- 172.21.15.118:0/1805842707 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7200c0cf0 0x7fe720069d40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 -- 172.21.15.118:0/1805842707 >> 172.21.15.118:0/1805842707 conn(0x7fe720067e60 msgr2=0x7fe7200cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 -- 172.21.15.118:0/1805842707 shutdown_connections 2024-08-18T13:42:11.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 -- 172.21.15.118:0/1805842707 wait complete. 2024-08-18T13:42:11.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 Processor -- start 2024-08-18T13:42:11.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.599+0000 7fe7260a8700 1 -- start start 2024-08-18T13:42:11.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe720068260 0x7fe720165b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:11.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7200897f0 0x7fe7201543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:11.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7200c0cf0 0x7fe7200b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:11.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720153c00 con 0x7fe720068260 2024-08-18T13:42:11.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7201660a0 con 0x7fe7200c0cf0 2024-08-18T13:42:11.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200af2b0 con 0x7fe7200897f0 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe720068260 0x7fe720165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe720068260 0x7fe720165b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41624/0 (socket says 172.21.15.118:41624) 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 -- 172.21.15.118:0/4040771732 learned_addr learned my addr 172.21.15.118:0/4040771732 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71effd700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7200897f0 0x7fe7201543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71ffff700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7200c0cf0 0x7fe7200b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 -- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7200897f0 msgr2=0x7fe7201543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7200897f0 0x7fe7201543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 -- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7200c0cf0 msgr2=0x7fe7200b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7200c0cf0 0x7fe7200b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 -- 172.21.15.118:0/4040771732 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe714009420 con 0x7fe720068260 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe71f7fe700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe720068260 0x7fe720165b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe714009f80 tx=0x7fe714017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7077fe700 1 -- 172.21.15.118:0/4040771732 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71401bc30 con 0x7fe720068260 2024-08-18T13:42:11.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7077fe700 1 -- 172.21.15.118:0/4040771732 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71401bdd0 con 0x7fe720068260 2024-08-18T13:42:11.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe720153640 con 0x7fe720068260 2024-08-18T13:42:11.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7077fe700 1 -- 172.21.15.118:0/4040771732 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714011670 con 0x7fe720068260 2024-08-18T13:42:11.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.603+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7200aec60 con 0x7fe720068260 2024-08-18T13:42:11.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.607+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7200aeac0 con 0x7fe720068260 2024-08-18T13:42:11.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.607+0000 7fe7077fe700 1 -- 172.21.15.118:0/4040771732 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe7140119e0 con 0x7fe720068260 2024-08-18T13:42:11.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.607+0000 7fe7077fe700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe70c060860 0x7fe70c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:11.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.607+0000 7fe7077fe700 1 -- 172.21.15.118:0/4040771732 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6715+0+0 (secure 0 0 0) 0x7fe71407e490 con 0x7fe720068260 2024-08-18T13:42:11.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.607+0000 7fe71effd700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe70c060860 0x7fe70c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:11.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.607+0000 7fe71effd700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe70c060860 0x7fe70c062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe720159920 tx=0x7fe7100091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:11.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.615+0000 7fe7077fe700 1 -- 172.21.15.118:0/4040771732 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe7200aeac0 con 0x7fe720068260 2024-08-18T13:42:11.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.819+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7200aeac0 con 0x7fe720068260 2024-08-18T13:42:11.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.819+0000 7fe7077fe700 1 -- 172.21.15.118:0/4040771732 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7fe7200aeac0 con 0x7fe720068260 2024-08-18T13:42:11.825 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:11.825 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:11.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe70c060860 msgr2=0x7fe70c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:11.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe70c060860 0x7fe70c062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe720159920 tx=0x7fe7100091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe720068260 msgr2=0x7fe720165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe720068260 0x7fe720165b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe714009f80 tx=0x7fe714017000 comp rx=0 tx=0).stop 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 shutdown_connections 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe70c060860 0x7fe70c062d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7200c0cf0 0x7fe7200b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7200897f0 0x7fe7201543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 --2- 172.21.15.118:0/4040771732 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe720068260 0x7fe720165b60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 >> 172.21.15.118:0/4040771732 conn(0x7fe720067e60 msgr2=0x7fe72014db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 shutdown_connections 2024-08-18T13:42:11.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:11.823+0000 7fe7260a8700 1 -- 172.21.15.118:0/4040771732 wait complete. 2024-08-18T13:42:11.829 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:42:11.839 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:42:12.263 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:42:12.266 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:42:13.839 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:13.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 Processor -- start 2024-08-18T13:42:13.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 -- start start 2024-08-18T13:42:13.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7feec414d6b0 0x7feec40c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:13.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec40c36d0 0x7feec40673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:13.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feec4067910 0x7feec406b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:13.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec40c3eb0 con 0x7feec414d6b0 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec41577f0 con 0x7feec4067910 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec9652700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec4157360 con 0x7feec40c36d0 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7feec414d6b0 0x7feec40c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7feec414d6b0 0x7feec40c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59636/0 (socket says 172.21.15.118:59636) 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 -- 172.21.15.118:0/2109868259 learned_addr learned my addr 172.21.15.118:0/2109868259 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec37fe700 1 --2- 172.21.15.118:0/2109868259 >> v2:172.21.15.178:3300/0 conn(0x7feec4067910 0x7feec406b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec27fc700 1 --2- 172.21.15.118:0/2109868259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec40c36d0 0x7feec40673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 -- 172.21.15.118:0/2109868259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec40c36d0 msgr2=0x7feec40673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 --2- 172.21.15.118:0/2109868259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec40c36d0 0x7feec40673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 -- 172.21.15.118:0/2109868259 >> v2:172.21.15.178:3300/0 conn(0x7feec4067910 msgr2=0x7feec406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:13.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 --2- 172.21.15.118:0/2109868259 >> v2:172.21.15.178:3300/0 conn(0x7feec4067910 0x7feec406b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:13.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 -- 172.21.15.118:0/2109868259 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40c3bd0 con 0x7feec414d6b0 2024-08-18T13:42:13.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.951+0000 7feec2ffd700 1 --2- 172.21.15.118:0/2109868259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec414d6b0 0x7feec40c3190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feec4150f10 tx=0x7feeb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8727a4abdabbb240 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec1ffb700 1 -- 172.21.15.118:0/2109868259 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800b640 con 0x7feec414d6b0 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec1ffb700 1 -- 172.21.15.118:0/2109868259 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb800b7a0 con 0x7feec414d6b0 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec1ffb700 1 -- 172.21.15.118:0/2109868259 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800bb40 con 0x7feec414d6b0 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- 172.21.15.118:0/2109868259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec414d6b0 msgr2=0x7feec40c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 --2- 172.21.15.118:0/2109868259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec414d6b0 0x7feec40c3190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feec4150f10 tx=0x7feeb8009250 comp rx=0 tx=0).stop 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- 172.21.15.118:0/2109868259 shutdown_connections 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 --2- 172.21.15.118:0/2109868259 >> v2:172.21.15.178:3300/0 conn(0x7feec4067910 0x7feec406b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 --2- 172.21.15.118:0/2109868259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec40c36d0 0x7feec40673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 --2- 172.21.15.118:0/2109868259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec414d6b0 0x7feec40c3190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:13.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- 172.21.15.118:0/2109868259 >> 172.21.15.118:0/2109868259 conn(0x7feec40c7c40 msgr2=0x7feec40c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:13.960 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- 172.21.15.118:0/2109868259 shutdown_connections 2024-08-18T13:42:13.960 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- 172.21.15.118:0/2109868259 wait complete. 2024-08-18T13:42:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 Processor -- start 2024-08-18T13:42:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- start start 2024-08-18T13:42:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec4067910 0x7feec40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec40c36d0 0x7feec40c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:13.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec414d6b0 0x7feec40c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec414f800 con 0x7feec4067910 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4152b80 con 0x7feec40c36d0 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec9652700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec414f2f0 con 0x7feec414d6b0 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec2ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec4067910 0x7feec40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec2ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec4067910 0x7feec40acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59648/0 (socket says 172.21.15.118:59648) 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.955+0000 7feec2ffd700 1 -- 172.21.15.118:0/4126781705 learned_addr learned my addr 172.21.15.118:0/4126781705 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec27fc700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec40c36d0 0x7feec40c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec37fe700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec414d6b0 0x7feec40c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec2ffd700 1 -- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec414d6b0 msgr2=0x7feec40c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec2ffd700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec414d6b0 0x7feec40c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec2ffd700 1 -- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec40c36d0 msgr2=0x7feec40c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:13.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec2ffd700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec40c36d0 0x7feec40c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:13.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec2ffd700 1 -- 172.21.15.118:0/4126781705 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80094f0 con 0x7feec4067910 2024-08-18T13:42:13.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec2ffd700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec4067910 0x7feec40acdf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feeb80099b0 tx=0x7feeb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:13.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feeaaffd700 1 -- 172.21.15.118:0/4126781705 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8000d10 con 0x7feec4067910 2024-08-18T13:42:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feeaaffd700 1 -- 172.21.15.118:0/4126781705 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb800bb20 con 0x7feec4067910 2024-08-18T13:42:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec414f520 con 0x7feec4067910 2024-08-18T13:42:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feeaaffd700 1 -- 172.21.15.118:0/4126781705 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8011670 con 0x7feec4067910 2024-08-18T13:42:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec4168e10 con 0x7feec4067910 2024-08-18T13:42:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec40642f0 con 0x7feec4067910 2024-08-18T13:42:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.959+0000 7feeaaffd700 1 -- 172.21.15.118:0/4126781705 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7feeb8011a60 con 0x7feec4067910 2024-08-18T13:42:13.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.963+0000 7feeaaffd700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feeb0060860 0x7feeb0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:13.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.963+0000 7feeaaffd700 1 -- 172.21.15.118:0/4126781705 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6715+0+0 (secure 0 0 0) 0x7feeb807e8a0 con 0x7feec4067910 2024-08-18T13:42:13.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.963+0000 7feec27fc700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feeb0060860 0x7feeb0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:13.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.963+0000 7feec27fc700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feeb0060860 0x7feeb0062d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feec4162fe0 tx=0x7feeb40091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:13.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:13.967+0000 7feeaaffd700 1 -- 172.21.15.118:0/4126781705 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7feeb804e900 con 0x7feec4067910 2024-08-18T13:42:14.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feec40c2f70 con 0x7feec4067910 2024-08-18T13:42:14.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feeaaffd700 1 -- 172.21.15.118:0/4126781705 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6124 (secure 0 0 0) 0x7feeb8051f20 con 0x7feec4067910 2024-08-18T13:42:14.171 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:14.171 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:39:57.488947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.178:6837/2503344632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2503344632},{"type":"v1","addr":"172.21.15.178:6837","nonce":2503344632}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feeb0060860 msgr2=0x7feeb0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feeb0060860 0x7feeb0062d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feec4162fe0 tx=0x7feeb40091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec4067910 msgr2=0x7feec40acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec4067910 0x7feec40acdf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feeb80099b0 tx=0x7feeb8017000 comp rx=0 tx=0).stop 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 shutdown_connections 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feeb0060860 0x7feeb0062d10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec414d6b0 0x7feec40c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec40c36d0 0x7feec40c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 --2- 172.21.15.118:0/4126781705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feec4067910 0x7feec40acdf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:14.173 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 >> 172.21.15.118:0/4126781705 conn(0x7feec40c7c40 msgr2=0x7feec414c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:14.174 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.167+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 shutdown_connections 2024-08-18T13:42:14.174 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:14.171+0000 7feec9652700 1 -- 172.21.15.118:0/4126781705 wait complete. 2024-08-18T13:42:14.174 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 9 2024-08-18T13:42:14.184 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:42:16.185 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:16.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.299+0000 7f4f05c91700 1 Processor -- start 2024-08-18T13:42:16.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.299+0000 7f4f05c91700 1 -- start start 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4f000c2a20 0x7f4f00155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f0006ca70 0x7f4f0014bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f000c1e80 con 0x7f4f0006c6a0 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f001563d0 con 0x7f4f0006ca70 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f000c9690 con 0x7f4f000c2a20 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f0006ca70 0x7f4f0014bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f0006ca70 0x7f4f0014bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59656/0 (socket says 172.21.15.118:59656) 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 -- 172.21.15.118:0/843675949 learned_addr learned my addr 172.21.15.118:0/843675949 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:16.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4ef7fff700 1 --2- 172.21.15.118:0/843675949 >> v2:172.21.15.178:3300/0 conn(0x7f4f000c2a20 0x7f4f00155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 -- 172.21.15.118:0/843675949 >> v2:172.21.15.178:3300/0 conn(0x7f4f000c2a20 msgr2=0x7f4f00155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 --2- 172.21.15.118:0/843675949 >> v2:172.21.15.178:3300/0 conn(0x7f4f000c2a20 0x7f4f00155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 -- 172.21.15.118:0/843675949 >> v1:172.21.15.178:6789/0 conn(0x7f4f0006c6a0 legacy=0x7f4f000c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 -- 172.21.15.118:0/843675949 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000c1550 con 0x7f4f0006ca70 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05490700 1 --2- 172.21.15.118:0/843675949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006ca70 0x7f4f0014bb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f0006d8e0 tx=0x7f4ef8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d7da309b0c8c9c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4ef77fe700 1 -- 172.21.15.118:0/843675949 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800e070 con 0x7f4f0006ca70 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4ef77fe700 1 -- 172.21.15.118:0/843675949 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef800bab0 con 0x7f4f0006ca70 2024-08-18T13:42:16.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4ef77fe700 1 -- 172.21.15.118:0/843675949 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80092e0 con 0x7f4f0006ca70 2024-08-18T13:42:16.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- 172.21.15.118:0/843675949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006ca70 msgr2=0x7f4f0014bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 --2- 172.21.15.118:0/843675949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006ca70 0x7f4f0014bb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f0006d8e0 tx=0x7f4ef8009840 comp rx=0 tx=0).stop 2024-08-18T13:42:16.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- 172.21.15.118:0/843675949 shutdown_connections 2024-08-18T13:42:16.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 --2- 172.21.15.118:0/843675949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006ca70 0x7f4f0014bb80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 --2- 172.21.15.118:0/843675949 >> v2:172.21.15.178:3300/0 conn(0x7f4f000c2a20 0x7f4f00155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- 172.21.15.118:0/843675949 >> 172.21.15.118:0/843675949 conn(0x7f4f0006f4d0 msgr2=0x7f4f000c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:16.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- 172.21.15.118:0/843675949 shutdown_connections 2024-08-18T13:42:16.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.303+0000 7f4f05c91700 1 -- 172.21.15.118:0/843675949 wait complete. 2024-08-18T13:42:16.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 Processor -- start 2024-08-18T13:42:16.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 -- start start 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006c6a0 0x7f4f000c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f0006ca70 0x7f4f000c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f000c2a20 0x7f4f00142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0014e740 con 0x7f4f0006c6a0 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0014f920 con 0x7f4f000c2a20 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05c91700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f00142650 con 0x7f4f0006ca70 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006c6a0 0x7f4f000c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006c6a0 0x7f4f000c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59664/0 (socket says 172.21.15.118:59664) 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 -- 172.21.15.118:0/3209025224 learned_addr learned my addr 172.21.15.118:0/3209025224 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4ef7fff700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f0006ca70 0x7f4f000c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f05490700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f000c2a20 0x7f4f00142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 -- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f0006ca70 msgr2=0x7f4f000c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f0006ca70 0x7f4f000c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 -- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f000c2a20 msgr2=0x7f4f00142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f000c2a20 0x7f4f00142ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.307+0000 7f4f04c8f700 1 -- 172.21.15.118:0/3209025224 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8009ae0 con 0x7f4f0006c6a0 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4f04c8f700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006c6a0 0x7f4f000c7950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00c5c0 tx=0x7f4efc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4ef4ff9700 1 -- 172.21.15.118:0/3209025224 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc011590 con 0x7f4f0006c6a0 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4ef4ff9700 1 -- 172.21.15.118:0/3209025224 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc011710 con 0x7f4f0006c6a0 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f00067f60 con 0x7f4f0006c6a0 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4ef4ff9700 1 -- 172.21.15.118:0/3209025224 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc011ad0 con 0x7f4f0006c6a0 2024-08-18T13:42:16.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f001422e0 con 0x7f4f0006c6a0 2024-08-18T13:42:16.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f00142140 con 0x7f4f0006c6a0 2024-08-18T13:42:16.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4ef4ff9700 1 -- 172.21.15.118:0/3209025224 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4efc01b3b0 con 0x7f4f0006c6a0 2024-08-18T13:42:16.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4ef4ff9700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eec060860 0x7f4eec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4ef4ff9700 1 -- 172.21.15.118:0/3209025224 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f4efc07c4f0 con 0x7f4f0006c6a0 2024-08-18T13:42:16.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.311+0000 7f4ef7fff700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eec060860 0x7f4eec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.315+0000 7f4ef7fff700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eec060860 0x7f4eec062d10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f001563a0 tx=0x7f4ef00091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:16.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.319+0000 7f4ef4ff9700 1 -- 172.21.15.118:0/3209025224 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4f00142140 con 0x7f4f0006c6a0 2024-08-18T13:42:16.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.519+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f00142140 con 0x7f4f0006c6a0 2024-08-18T13:42:16.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.519+0000 7f4ef4ff9700 1 -- 172.21.15.118:0/3209025224 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f4f00142140 con 0x7f4f0006c6a0 2024-08-18T13:42:16.524 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:16.524 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:15.511575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:16.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eec060860 msgr2=0x7f4eec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eec060860 0x7f4eec062d10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f001563a0 tx=0x7f4ef00091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006c6a0 msgr2=0x7f4f000c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006c6a0 0x7f4f000c7950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00c5c0 tx=0x7f4efc00c8d0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 shutdown_connections 2024-08-18T13:42:16.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eec060860 0x7f4eec062d10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f000c2a20 0x7f4f00142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f0006ca70 0x7f4f000c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 --2- 172.21.15.118:0/3209025224 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f0006c6a0 0x7f4f000c7950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 >> 172.21.15.118:0/3209025224 conn(0x7f4f0006f4d0 msgr2=0x7f4f000c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:16.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 shutdown_connections 2024-08-18T13:42:16.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.523+0000 7f4f05c91700 1 -- 172.21.15.118:0/3209025224 wait complete. 2024-08-18T13:42:16.528 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 10 2024-08-18T13:42:16.538 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2024-08-18T13:42:16.538 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-08-18T13:42:16.538 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:16.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.639+0000 7f9e38a0c700 1 Processor -- start 2024-08-18T13:42:16.645 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.639+0000 7f9e38a0c700 1 -- start start 2024-08-18T13:42:16.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9e340c0b70 0x7f9e340c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9e340c0f40 0x7f9e3406a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e34068e00 con 0x7f9e34155c80 2024-08-18T13:42:16.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340690a0 con 0x7f9e340c0b70 2024-08-18T13:42:16.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e34069340 con 0x7f9e340c0f40 2024-08-18T13:42:16.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e33fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f9e34155c80 0x7f9e340cc2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:39730/0 (socket says 172.21.15.118:39730) 2024-08-18T13:42:16.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e33fff700 1 -- 172.21.15.118:0/586373402 learned_addr learned my addr 172.21.15.118:0/586373402 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:16.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e337fe700 1 --2- 172.21.15.118:0/586373402 >> v2:172.21.15.118:3300/0 conn(0x7f9e340c0b70 0x7f9e340c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e32ffd700 1 --2- 172.21.15.118:0/586373402 >> v2:172.21.15.178:3300/0 conn(0x7f9e340c0f40 0x7f9e3406a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e327fc700 1 -- 172.21.15.118:0/586373402 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741254049 0 0) 0x7f9e34068e00 con 0x7f9e34155c80 2024-08-18T13:42:16.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e327fc700 1 -- 172.21.15.118:0/586373402 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e20003570 con 0x7f9e34155c80 2024-08-18T13:42:16.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e337fe700 1 -- 172.21.15.118:0/586373402 >> v2:172.21.15.178:3300/0 conn(0x7f9e340c0f40 msgr2=0x7f9e3406a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:16.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e337fe700 1 --2- 172.21.15.118:0/586373402 >> v2:172.21.15.178:3300/0 conn(0x7f9e340c0f40 0x7f9e3406a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e337fe700 1 -- 172.21.15.118:0/586373402 >> v1:172.21.15.118:6789/0 conn(0x7f9e34155c80 legacy=0x7f9e340cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e337fe700 1 -- 172.21.15.118:0/586373402 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340695e0 con 0x7f9e340c0b70 2024-08-18T13:42:16.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e337fe700 1 --2- 172.21.15.118:0/586373402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0b70 0x7f9e340c6510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e340b1ba0 tx=0x7f9e240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5b8bb8aaf88ad1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:16.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e327fc700 1 -- 172.21.15.118:0/586373402 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24011630 con 0x7f9e340c0b70 2024-08-18T13:42:16.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e327fc700 1 -- 172.21.15.118:0/586373402 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24011790 con 0x7f9e340c0b70 2024-08-18T13:42:16.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e327fc700 1 -- 172.21.15.118:0/586373402 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24011b30 con 0x7f9e340c0b70 2024-08-18T13:42:16.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- 172.21.15.118:0/586373402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0b70 msgr2=0x7f9e340c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/586373402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0b70 0x7f9e340c6510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e340b1ba0 tx=0x7f9e240091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- 172.21.15.118:0/586373402 shutdown_connections 2024-08-18T13:42:16.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/586373402 >> v2:172.21.15.178:3300/0 conn(0x7f9e340c0f40 0x7f9e3406a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/586373402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0b70 0x7f9e340c6510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- 172.21.15.118:0/586373402 >> 172.21.15.118:0/586373402 conn(0x7f9e34067e50 msgr2=0x7f9e340cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:16.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- 172.21.15.118:0/586373402 shutdown_connections 2024-08-18T13:42:16.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.643+0000 7f9e38a0c700 1 -- 172.21.15.118:0/586373402 wait complete. 2024-08-18T13:42:16.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 Processor -- start 2024-08-18T13:42:16.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 -- start start 2024-08-18T13:42:16.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e340c0b70 0x7f9e34151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0f40 0x7f9e340b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e34155c80 0x7f9e341484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3414c500 con 0x7f9e340c0f40 2024-08-18T13:42:16.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34151b60 con 0x7f9e34155c80 2024-08-18T13:42:16.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3414bf40 con 0x7f9e340c0b70 2024-08-18T13:42:16.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0f40 0x7f9e340b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0f40 0x7f9e340b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59686/0 (socket says 172.21.15.118:59686) 2024-08-18T13:42:16.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 -- 172.21.15.118:0/2183505580 learned_addr learned my addr 172.21.15.118:0/2183505580 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:16.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e33fff700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e34155c80 0x7f9e341484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e337fe700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e340c0b70 0x7f9e34151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 -- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e340c0b70 msgr2=0x7f9e34151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e340c0b70 0x7f9e34151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 -- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e34155c80 msgr2=0x7f9e341484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e34155c80 0x7f9e341484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 -- 172.21.15.118:0/2183505580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24009420 con 0x7f9e340c0f40 2024-08-18T13:42:16.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e32ffd700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0f40 0x7f9e340b11a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00ec80 tx=0x7f9e1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:16.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e2b7fe700 1 -- 172.21.15.118:0/2183505580 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00ca60 con 0x7f9e340c0f40 2024-08-18T13:42:16.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e2b7fe700 1 -- 172.21.15.118:0/2183505580 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00cbc0 con 0x7f9e340c0f40 2024-08-18T13:42:16.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3414c220 con 0x7f9e340c0f40 2024-08-18T13:42:16.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e2b7fe700 1 -- 172.21.15.118:0/2183505580 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c013470 con 0x7f9e340c0f40 2024-08-18T13:42:16.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e34165340 con 0x7f9e340c0f40 2024-08-18T13:42:16.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.647+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e180052b0 con 0x7f9e340c0f40 2024-08-18T13:42:16.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.651+0000 7f9e2b7fe700 1 -- 172.21.15.118:0/2183505580 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9e1c013820 con 0x7f9e340c0f40 2024-08-18T13:42:16.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.651+0000 7f9e2b7fe700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e14060860 0x7f9e14062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:16.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.651+0000 7f9e2b7fe700 1 -- 172.21.15.118:0/2183505580 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f9e1c07d300 con 0x7f9e340c0f40 2024-08-18T13:42:16.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.651+0000 7f9e337fe700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e14060860 0x7f9e14062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:16.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.651+0000 7f9e337fe700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e14060860 0x7f9e14062d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e340b1ba0 tx=0x7f9e24017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:16.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.655+0000 7f9e2b7fe700 1 -- 172.21.15.118:0/2183505580 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9e1c04d340 con 0x7f9e340c0f40 2024-08-18T13:42:16.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.851+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e18005560 con 0x7f9e340c0f40 2024-08-18T13:42:16.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e2b7fe700 1 -- 172.21.15.118:0/2183505580 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f9e1c050960 con 0x7f9e340c0f40 2024-08-18T13:42:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:16.858 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:15.511575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e14060860 msgr2=0x7f9e14062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e14060860 0x7f9e14062d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e340b1ba0 tx=0x7f9e24017000 comp rx=0 tx=0).stop 2024-08-18T13:42:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0f40 msgr2=0x7f9e340b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0f40 0x7f9e340b11a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00ec80 tx=0x7f9e1c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 shutdown_connections 2024-08-18T13:42:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9e14060860 0x7f9e14062d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e34155c80 0x7f9e341484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e340c0f40 0x7f9e340b11a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 --2- 172.21.15.118:0/2183505580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e340c0b70 0x7f9e34151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 >> 172.21.15.118:0/2183505580 conn(0x7f9e34067e50 msgr2=0x7f9e3414ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 shutdown_connections 2024-08-18T13:42:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:16.855+0000 7f9e38a0c700 1 -- 172.21.15.118:0/2183505580 wait complete. 2024-08-18T13:42:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 10 2024-08-18T13:42:16.874 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 10, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 10, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4306": { "addr": "172.21.15.178:6835/1590569622", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6832", "nonce": 1590569622, "type": "v2" }, { "addr": "172.21.15.178:6835", "nonce": 1590569622, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4306, "incarnation": 10, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:42:15.511575+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4306 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:42:18.875 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:18.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7be8f8700 1 Processor -- start 2024-08-18T13:42:18.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7be8f8700 1 -- start start 2024-08-18T13:42:18.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7be8f8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b80c3d50 0x7fa7b80c4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:18.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7be8f8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b80c3020 con 0x7fa7b80be710 2024-08-18T13:42:18.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7be8f8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b80c38c0 con 0x7fa7b80be340 2024-08-18T13:42:18.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7be8f8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80beb10 con 0x7fa7b80c3d50 2024-08-18T13:42:18.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fa7b80be710 0x7fa7b806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:39742/0 (socket says 172.21.15.118:39742) 2024-08-18T13:42:18.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 -- 172.21.15.118:0/3718752679 learned_addr learned my addr 172.21.15.118:0/3718752679 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:18.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7bce95700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b80c3d50 0x7fa7b80c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172246115 0 0) 0x7fa7b80c3020 con 0x7fa7b80be710 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a0003570 con 0x7fa7b80be710 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892693669 0 0) 0x7fa7b80c38c0 con 0x7fa7b80be340 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b80c3020 con 0x7fa7b80be340 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409280535 0 0) 0x7fa7a0003570 con 0x7fa7b80be710 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b80c3d50 msgr2=0x7fa7b80c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b80c3d50 0x7fa7b80c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 >> v1:172.21.15.178:6789/0 conn(0x7fa7b80be340 legacy=0x7fa7b815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b806e3b0 con 0x7fa7b80be710 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fa7b00017f0 con 0x7fa7b80be710 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 >> v1:172.21.15.118:6789/0 conn(0x7fa7b80be710 legacy=0x7fa7b806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7a0003c20 0x7fa7a0003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7a00051c0 0x7fa7a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80beb10 con 0x7fa7a0004810 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a000bfa0 con 0x7fa7a00051c0 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a000c160 con 0x7fa7a0003c20 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7bce95700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b80c3d50 0x7fa7b80c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b7fff700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7a0003c20 0x7fa7a0003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 -- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7a0003c20 msgr2=0x7fa7a0003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7bce95700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7a00051c0 0x7fa7a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7a0003c20 0x7fa7a0003ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 -- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7a00051c0 msgr2=0x7fa7a000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7a00051c0 0x7fa7a000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 -- 172.21.15.118:0/3718752679 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a0003570 con 0x7fa7a0004810 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7bce95700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7a00051c0 0x7fa7a000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.991+0000 7fa7b77fe700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7a0004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000abc0 tx=0x7fa7b806f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b5efce605168e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000c5b0 con 0x7fa7a0004810 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b000d040 con 0x7fa7a0004810 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7b6ffd700 1 -- 172.21.15.118:0/3718752679 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b000c980 con 0x7fa7a0004810 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 -- 172.21.15.118:0/3718752679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 msgr2=0x7fa7a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 --2- 172.21.15.118:0/3718752679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7a0004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000abc0 tx=0x7fa7b806f310 comp rx=0 tx=0).stop 2024-08-18T13:42:19.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7bce95700 1 -- 172.21.15.118:0/3718752679 reap_dead start 2024-08-18T13:42:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 -- 172.21.15.118:0/3718752679 shutdown_connections 2024-08-18T13:42:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 -- 172.21.15.118:0/3718752679 >> 172.21.15.118:0/3718752679 conn(0x7fa7b80c7c40 msgr2=0x7fa7b80c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 -- 172.21.15.118:0/3718752679 shutdown_connections 2024-08-18T13:42:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 -- 172.21.15.118:0/3718752679 wait complete. 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 Processor -- start 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.995+0000 7fa7be8f8700 1 -- start start 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7b8154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b815da30 0x7fa7b81591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b80c18f0 0x7fa7b8159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8154810 con 0x7fa7a0004810 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8155660 con 0x7fa7b80c18f0 2024-08-18T13:42:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8155920 con 0x7fa7b815da30 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7b8154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7b8154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59698/0 (socket says 172.21.15.118:59698) 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 -- 172.21.15.118:0/2388375719 learned_addr learned my addr 172.21.15.118:0/2388375719 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7bce95700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b80c18f0 0x7fa7b8159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b77fe700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b815da30 0x7fa7b81591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 -- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b815da30 msgr2=0x7fa7b81591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b815da30 0x7fa7b81591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 -- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b80c18f0 msgr2=0x7fa7b8159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b80c18f0 0x7fa7b8159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 -- 172.21.15.118:0/2388375719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b816a630 con 0x7fa7a0004810 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7b7fff700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7b8154ca0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00d830 tx=0x7fa7ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa79ffff700 1 -- 172.21.15.118:0/2388375719 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac012600 con 0x7fa7a0004810 2024-08-18T13:42:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa79ffff700 1 -- 172.21.15.118:0/2388375719 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac012760 con 0x7fa7a0004810 2024-08-18T13:42:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b816a910 con 0x7fa7a0004810 2024-08-18T13:42:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa79ffff700 1 -- 172.21.15.118:0/2388375719 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac012b00 con 0x7fa7a0004810 2024-08-18T13:42:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b81544a0 con 0x7fa7a0004810 2024-08-18T13:42:19.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b80ada20 con 0x7fa7a0004810 2024-08-18T13:42:19.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:18.999+0000 7fa79ffff700 1 -- 172.21.15.118:0/2388375719 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa7ac01a3b0 con 0x7fa7a0004810 2024-08-18T13:42:19.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.003+0000 7fa79ffff700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7a4060860 0x7fa7a4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:19.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.003+0000 7fa79ffff700 1 -- 172.21.15.118:0/2388375719 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7fa7ac07cfb0 con 0x7fa7a0004810 2024-08-18T13:42:19.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.003+0000 7fa7b77fe700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7a4060860 0x7fa7a4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:19.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.003+0000 7fa7b77fe700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7a4060860 0x7fa7a4062d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7b815de00 tx=0x7fa7b0016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:19.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.015+0000 7fa79ffff700 1 -- 172.21.15.118:0/2388375719 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa7ac0ad480 con 0x7fa7a0004810 2024-08-18T13:42:19.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.215+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7b80642f0 con 0x7fa7a0004810 2024-08-18T13:42:19.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa79ffff700 1 -- 172.21.15.118:0/2388375719 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+5369 (secure 0 0 0) 0x7fa7b80ada20 con 0x7fa7a0004810 2024-08-18T13:42:19.221 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:19.221 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:15.511575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7a4060860 msgr2=0x7fa7a4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7a4060860 0x7fa7a4062d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7b815de00 tx=0x7fa7b0016000 comp rx=0 tx=0).stop 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 msgr2=0x7fa7b8154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7b8154ca0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00d830 tx=0x7fa7ac009040 comp rx=0 tx=0).stop 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 shutdown_connections 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7a4060860 0x7fa7a4062d10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7b80c18f0 0x7fa7b8159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7b815da30 0x7fa7b81591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 --2- 172.21.15.118:0/2388375719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7a0004810 0x7fa7b8154ca0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 >> 172.21.15.118:0/2388375719 conn(0x7fa7b80c7c40 msgr2=0x7fa7b815a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 shutdown_connections 2024-08-18T13:42:19.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:19.219+0000 7fa7be8f8700 1 -- 172.21.15.118:0/2388375719 wait complete. 2024-08-18T13:42:19.225 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 10 2024-08-18T13:42:19.241 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:21.243 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:21.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 Processor -- start 2024-08-18T13:42:21.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- start start 2024-08-18T13:42:21.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700be340 0x7f39700be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:21.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f397015a310 0x7f397006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:21.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39700c3020 con 0x7f39700c3d50 2024-08-18T13:42:21.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39700c38c0 con 0x7f397015a310 2024-08-18T13:42:21.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397006e3b0 con 0x7f39700be340 2024-08-18T13:42:21.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f397015a310 0x7f397006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:21.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f397015a310 0x7f397006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59708/0 (socket says 172.21.15.118:59708) 2024-08-18T13:42:21.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 -- 172.21.15.118:0/1406184799 learned_addr learned my addr 172.21.15.118:0/1406184799 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:21.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3975212700 1 --2- 172.21.15.118:0/1406184799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700be340 0x7f39700be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:21.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 -- 172.21.15.118:0/1406184799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700be340 msgr2=0x7f39700be710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:21.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 --2- 172.21.15.118:0/1406184799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700be340 0x7f39700be710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 -- 172.21.15.118:0/1406184799 >> v1:172.21.15.178:6789/0 conn(0x7f39700c3d50 legacy=0x7f397006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:21.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 -- 172.21.15.118:0/1406184799 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700673d0 con 0x7f397015a310 2024-08-18T13:42:21.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3974a11700 1 --2- 172.21.15.118:0/1406184799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f397006db20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3968009f20 tx=0x7f3968009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341e13f474e89aef server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:21.364 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3963fff700 1 -- 172.21.15.118:0/1406184799 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396800e070 con 0x7f397015a310 2024-08-18T13:42:21.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3963fff700 1 -- 172.21.15.118:0/1406184799 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396800ba80 con 0x7f397015a310 2024-08-18T13:42:21.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3963fff700 1 -- 172.21.15.118:0/1406184799 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39680092e0 con 0x7f397015a310 2024-08-18T13:42:21.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- 172.21.15.118:0/1406184799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 msgr2=0x7f397006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:21.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 --2- 172.21.15.118:0/1406184799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f397006db20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3968009f20 tx=0x7f3968009840 comp rx=0 tx=0).stop 2024-08-18T13:42:21.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- 172.21.15.118:0/1406184799 shutdown_connections 2024-08-18T13:42:21.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 --2- 172.21.15.118:0/1406184799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f397006db20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 --2- 172.21.15.118:0/1406184799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700be340 0x7f39700be710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- 172.21.15.118:0/1406184799 >> 172.21.15.118:0/1406184799 conn(0x7f39700c7c40 msgr2=0x7f39700c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:21.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- 172.21.15.118:0/1406184799 shutdown_connections 2024-08-18T13:42:21.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.355+0000 7f3977476700 1 -- 172.21.15.118:0/1406184799 wait complete. 2024-08-18T13:42:21.371 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.359+0000 7f3977476700 1 Processor -- start 2024-08-18T13:42:21.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.359+0000 7f3977476700 1 -- start start 2024-08-18T13:42:21.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39700be340 0x7f397016a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:21.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700c3d50 0x7f39700c1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:21.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f39700c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:21.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3970155570 con 0x7f397015a310 2024-08-18T13:42:21.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397015cf80 con 0x7f39700be340 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39701542f0 con 0x7f39700c3d50 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975a13700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f39700c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975212700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39700be340 0x7f397016a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3974a11700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700c3d50 0x7f39700c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3974a11700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700c3d50 0x7f39700c1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:42170/0 (socket says 172.21.15.118:42170) 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3974a11700 1 -- 172.21.15.118:0/2869309654 learned_addr learned my addr 172.21.15.118:0/2869309654 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975a13700 1 -- 172.21.15.118:0/2869309654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700c3d50 msgr2=0x7f39700c1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975a13700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700c3d50 0x7f39700c1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975a13700 1 -- 172.21.15.118:0/2869309654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39700be340 msgr2=0x7f397016a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975a13700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39700be340 0x7f397016a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975a13700 1 -- 172.21.15.118:0/2869309654 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968009ae0 con 0x7f397015a310 2024-08-18T13:42:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975212700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39700be340 0x7f397016a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:42:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3975a13700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f39700c0b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c4e0 tx=0x7f396c007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f39617fa700 1 -- 172.21.15.118:0/2869309654 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c00ce50 con 0x7f397015a310 2024-08-18T13:42:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f39617fa700 1 -- 172.21.15.118:0/2869309654 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c011590 con 0x7f397015a310 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f39617fa700 1 -- 172.21.15.118:0/2869309654 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c011950 con 0x7f397015a310 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397015d1d0 con 0x7f397015a310 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397015d790 con 0x7f397015a310 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.363+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39700642f0 con 0x7f397015a310 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.367+0000 7f39617fa700 1 -- 172.21.15.118:0/2869309654 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f396c011cc0 con 0x7f397015a310 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.367+0000 7f39617fa700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3954060860 0x7f3954062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.367+0000 7f39617fa700 1 -- 172.21.15.118:0/2869309654 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f396c07c160 con 0x7f397015a310 2024-08-18T13:42:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.367+0000 7f3975212700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3954060860 0x7f3954062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.367+0000 7f3975212700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3954060860 0x7f3954062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3964006fd0 tx=0x7f3964008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:21.377 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.375+0000 7f39617fa700 1 -- 172.21.15.118:0/2869309654 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f396c04c1a0 con 0x7f397015a310 2024-08-18T13:42:21.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.575+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3970154430 con 0x7f397015a310 2024-08-18T13:42:21.580 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.575+0000 7f39617fa700 1 -- 172.21.15.118:0/2869309654 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f396c04f7c0 con 0x7f397015a310 2024-08-18T13:42:21.581 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:21.581 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:15.511575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:21.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3954060860 msgr2=0x7f3954062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:21.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3954060860 0x7f3954062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3964006fd0 tx=0x7f3964008040 comp rx=0 tx=0).stop 2024-08-18T13:42:21.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 msgr2=0x7f39700c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:21.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f39700c0b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c4e0 tx=0x7f396c007e60 comp rx=0 tx=0).stop 2024-08-18T13:42:21.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 shutdown_connections 2024-08-18T13:42:21.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3954060860 0x7f3954062d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f397015a310 0x7f39700c0b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39700c3d50 0x7f39700c1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 --2- 172.21.15.118:0/2869309654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39700be340 0x7f397016a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:21.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 >> 172.21.15.118:0/2869309654 conn(0x7f39700c7c40 msgr2=0x7f3970152270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:21.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 shutdown_connections 2024-08-18T13:42:21.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:21.579+0000 7f3977476700 1 -- 172.21.15.118:0/2869309654 wait complete. 2024-08-18T13:42:21.585 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 10 2024-08-18T13:42:21.595 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:23.596 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:23.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.703+0000 7fa67acbb700 1 Processor -- start 2024-08-18T13:42:23.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 -- start start 2024-08-18T13:42:23.710 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa6740adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6740c02b0 0x7fa67415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740bfe20 con 0x7fa674158720 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6740ad3f0 con 0x7fa6740c02b0 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740bef50 con 0x7fa67414f330 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6740c02b0 0x7fa67415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa6740c02b0 0x7fa67415fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59728/0 (socket says 172.21.15.118:59728) 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3509155971 learned_addr learned my addr 172.21.15.118:0/3509155971 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:23.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa679cb9700 1 --2- 172.21.15.118:0/3509155971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa6740adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3509155971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 msgr2=0x7fa6740adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 --2- 172.21.15.118:0/3509155971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa6740adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3509155971 >> v1:172.21.15.178:6789/0 conn(0x7fa674158720 legacy=0x7fa67406bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3509155971 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740bf210 con 0x7fa6740c02b0 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa678cb7700 1 -- 172.21.15.118:0/3509155971 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604825223 0 0) 0x7fa6740bfe20 con 0x7fa674158720 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67a4ba700 1 --2- 172.21.15.118:0/3509155971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6740c02b0 0x7fa67415fac0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009f20 tx=0x7fa66c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38802fa5eda85ca server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa678cb7700 1 -- 172.21.15.118:0/3509155971 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c00e070 con 0x7fa6740c02b0 2024-08-18T13:42:23.713 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa678cb7700 1 -- 172.21.15.118:0/3509155971 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c00ba60 con 0x7fa6740c02b0 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa678cb7700 1 -- 172.21.15.118:0/3509155971 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c0092e0 con 0x7fa6740c02b0 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 -- 172.21.15.118:0/3509155971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6740c02b0 msgr2=0x7fa67415fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3509155971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6740c02b0 0x7fa67415fac0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009f20 tx=0x7fa66c009840 comp rx=0 tx=0).stop 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 -- 172.21.15.118:0/3509155971 shutdown_connections 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3509155971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6740c02b0 0x7fa67415fac0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.707+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3509155971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa6740adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 -- 172.21.15.118:0/3509155971 >> 172.21.15.118:0/3509155971 conn(0x7fa6740ca6d0 msgr2=0x7fa67406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 -- 172.21.15.118:0/3509155971 shutdown_connections 2024-08-18T13:42:23.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 -- 172.21.15.118:0/3509155971 wait complete. 2024-08-18T13:42:23.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 Processor -- start 2024-08-18T13:42:23.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 -- start start 2024-08-18T13:42:23.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6740c02b0 0x7fa674168890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:23.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa674069120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:23.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa674158720 0x7fa674069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:23.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa674152d90 con 0x7fa674158720 2024-08-18T13:42:23.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67406a8d0 con 0x7fa6740c02b0 2024-08-18T13:42:23.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67acbb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa674169310 con 0x7fa67414f330 2024-08-18T13:42:23.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa674158720 0x7fa674069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa674158720 0x7fa674069660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:59740/0 (socket says 172.21.15.118:59740) 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3792455790 learned_addr learned my addr 172.21.15.118:0/3792455790 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa679cb9700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6740c02b0 0x7fa674168890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa6794b8700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa674069120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 msgr2=0x7fa674069120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa674069120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6740c02b0 msgr2=0x7fa674168890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6740c02b0 0x7fa674168890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.711+0000 7fa67a4ba700 1 -- 172.21.15.118:0/3792455790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c009ae0 con 0x7fa674158720 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa67a4ba700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa674158720 0x7fa674069660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa674064dd0 tx=0x7fa66c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa669ffb700 1 -- 172.21.15.118:0/3792455790 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c0032f0 con 0x7fa674158720 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa669ffb700 1 -- 172.21.15.118:0/3792455790 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c008e10 con 0x7fa674158720 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67406b0c0 con 0x7fa674158720 2024-08-18T13:42:23.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa669ffb700 1 -- 172.21.15.118:0/3792455790 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c01d670 con 0x7fa674158720 2024-08-18T13:42:23.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa674152fd0 con 0x7fa674158720 2024-08-18T13:42:23.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6741531c0 con 0x7fa674158720 2024-08-18T13:42:23.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa669ffb700 1 -- 172.21.15.118:0/3792455790 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa66c003040 con 0x7fa674158720 2024-08-18T13:42:23.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.715+0000 7fa669ffb700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa658060860 0x7fa658062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:23.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.719+0000 7fa679cb9700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa658060860 0x7fa658062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:23.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.719+0000 7fa669ffb700 1 -- 172.21.15.118:0/3792455790 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7fa66c07ebb0 con 0x7fa674158720 2024-08-18T13:42:23.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.719+0000 7fa679cb9700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa658060860 0x7fa658062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa664009db0 tx=0x7fa6640091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:23.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.727+0000 7fa669ffb700 1 -- 172.21.15.118:0/3792455790 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa66c04ebf0 con 0x7fa674158720 2024-08-18T13:42:23.935 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.931+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa67406af20 con 0x7fa674158720 2024-08-18T13:42:23.936 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.931+0000 7fa669ffb700 1 -- 172.21.15.118:0/3792455790 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+5373 (secure 0 0 0) 0x7fa67406af20 con 0x7fa674158720 2024-08-18T13:42:23.936 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:23.936 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:21.620886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:reconnect","state_seq":45,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:23.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa658060860 msgr2=0x7fa658062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:23.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa658060860 0x7fa658062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa664009db0 tx=0x7fa6640091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa674158720 msgr2=0x7fa674069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:23.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa674158720 0x7fa674069660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa674064dd0 tx=0x7fa66c018000 comp rx=0 tx=0).stop 2024-08-18T13:42:23.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 shutdown_connections 2024-08-18T13:42:23.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa658060860 0x7fa658062d10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa674158720 0x7fa674069660 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa67414f330 0x7fa674069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 --2- 172.21.15.118:0/3792455790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6740c02b0 0x7fa674168890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:23.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 >> 172.21.15.118:0/3792455790 conn(0x7fa6740ca6d0 msgr2=0x7fa6740c87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:23.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 shutdown_connections 2024-08-18T13:42:23.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:23.935+0000 7fa67acbb700 1 -- 172.21.15.118:0/3792455790 wait complete. 2024-08-18T13:42:23.941 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 11 2024-08-18T13:42:23.951 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:25.952 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:26.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.059+0000 7f39a759e700 1 Processor -- start 2024-08-18T13:42:26.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.059+0000 7f39a759e700 1 -- start start 2024-08-18T13:42:26.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f39a814f7e0 0x7f39a80c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:26.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a80c1c80 con 0x7f39a814f410 2024-08-18T13:42:26.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a80c18e0 con 0x7f39a80abea0 2024-08-18T13:42:26.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a80c8180 con 0x7f39a814f7e0 2024-08-18T13:42:26.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a5d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f39a814f7e0 0x7f39a80c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:26.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a659c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f39a814f410 0x7f39a80ab790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36656/0 (socket says 172.21.15.118:36656) 2024-08-18T13:42:26.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a659c700 1 -- 172.21.15.118:0/966270244 learned_addr learned my addr 172.21.15.118:0/966270244 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:26.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a5d9b700 1 -- 172.21.15.118:0/966270244 >> v1:172.21.15.178:6789/0 conn(0x7f39a80abea0 legacy=0x7f39a80c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:26.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a5d9b700 1 -- 172.21.15.118:0/966270244 >> v1:172.21.15.118:6789/0 conn(0x7f39a814f410 legacy=0x7f39a80ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:26.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a5d9b700 1 -- 172.21.15.118:0/966270244 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80c09f0 con 0x7f39a814f7e0 2024-08-18T13:42:26.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a559a700 1 -- 172.21.15.118:0/966270244 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811204517 0 0) 0x7f39a80c1c80 con 0x7f39a814f410 2024-08-18T13:42:26.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a5d9b700 1 --2- 172.21.15.118:0/966270244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a80c7530 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39980098e0 tx=0x7f39980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe124fd6d02cc21 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a559a700 1 -- 172.21.15.118:0/966270244 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998009e80 con 0x7f39a814f7e0 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a559a700 1 -- 172.21.15.118:0/966270244 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800b8f0 con 0x7f39a814f7e0 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a559a700 1 -- 172.21.15.118:0/966270244 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800bc70 con 0x7f39a814f7e0 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- 172.21.15.118:0/966270244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 msgr2=0x7f39a80c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 --2- 172.21.15.118:0/966270244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a80c7530 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39980098e0 tx=0x7f39980091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- 172.21.15.118:0/966270244 shutdown_connections 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 --2- 172.21.15.118:0/966270244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a80c7530 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- 172.21.15.118:0/966270244 >> 172.21.15.118:0/966270244 conn(0x7f39a80c8620 msgr2=0x7f39a80c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- 172.21.15.118:0/966270244 shutdown_connections 2024-08-18T13:42:26.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.063+0000 7f39a759e700 1 -- 172.21.15.118:0/966270244 wait complete. 2024-08-18T13:42:26.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 Processor -- start 2024-08-18T13:42:26.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 -- start start 2024-08-18T13:42:26.073 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a80abea0 0x7f39a8155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:26.073 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39a814f410 0x7f39a8165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:26.073 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a8149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8165640 con 0x7f39a814f7e0 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8153230 con 0x7f39a814f410 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8157c80 con 0x7f39a80abea0 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a8149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a8149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51638/0 (socket says 172.21.15.118:51638) 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 -- 172.21.15.118:0/2494925701 learned_addr learned my addr 172.21.15.118:0/2494925701 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a5d9b700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39a814f410 0x7f39a8165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a659c700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a80abea0 0x7f39a8155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 -- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a80abea0 msgr2=0x7f39a8155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a80abea0 0x7f39a8155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 -- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39a814f410 msgr2=0x7f39a8165ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:26.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39a814f410 0x7f39a8165ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 -- 172.21.15.118:0/2494925701 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998009420 con 0x7f39a814f7e0 2024-08-18T13:42:26.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a5d9b700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39a814f410 0x7f39a8165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:26.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a6d9d700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a8149ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c940 tx=0x7f39a00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:26.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f399e7fc700 1 -- 172.21.15.118:0/2494925701 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a000a830 con 0x7f39a814f7e0 2024-08-18T13:42:26.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f399e7fc700 1 -- 172.21.15.118:0/2494925701 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a000a990 con 0x7f39a814f7e0 2024-08-18T13:42:26.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a81652f0 con 0x7f39a814f7e0 2024-08-18T13:42:26.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f399e7fc700 1 -- 172.21.15.118:0/2494925701 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a000ad50 con 0x7f39a814f7e0 2024-08-18T13:42:26.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.067+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a8153770 con 0x7f39a814f7e0 2024-08-18T13:42:26.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.071+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a8165480 con 0x7f39a814f7e0 2024-08-18T13:42:26.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.071+0000 7f399e7fc700 1 -- 172.21.15.118:0/2494925701 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f39a0012630 con 0x7f39a814f7e0 2024-08-18T13:42:26.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.071+0000 7f399e7fc700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3988060860 0x7f3988062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:26.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.071+0000 7f399e7fc700 1 -- 172.21.15.118:0/2494925701 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f39a007d2e0 con 0x7f39a814f7e0 2024-08-18T13:42:26.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.071+0000 7f39a659c700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3988060860 0x7f3988062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:26.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.071+0000 7f39a659c700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3988060860 0x7f3988062d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f39900011e0 tx=0x7f3990009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:26.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.083+0000 7f399e7fc700 1 -- 172.21.15.118:0/2494925701 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f39a8165480 con 0x7f39a814f7e0 2024-08-18T13:42:26.280 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.275+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39a8165480 con 0x7f39a814f7e0 2024-08-18T13:42:26.281 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f399e7fc700 1 -- 172.21.15.118:0/2494925701 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+5373 (secure 0 0 0) 0x7f39a8165480 con 0x7f39a814f7e0 2024-08-18T13:42:26.281 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:26.281 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:21.620886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:reconnect","state_seq":45,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:26.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3988060860 msgr2=0x7f3988062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:26.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3988060860 0x7f3988062d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f39900011e0 tx=0x7f3990009040 comp rx=0 tx=0).stop 2024-08-18T13:42:26.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 msgr2=0x7f39a8149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:26.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a8149ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c940 tx=0x7f39a00016f0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 shutdown_connections 2024-08-18T13:42:26.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3988060860 0x7f3988062d10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.284 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39a814f7e0 0x7f39a8149ff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39a814f410 0x7f39a8165ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 --2- 172.21.15.118:0/2494925701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39a80abea0 0x7f39a8155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:26.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 >> 172.21.15.118:0/2494925701 conn(0x7f39a80c8620 msgr2=0x7f39a80c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:26.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 shutdown_connections 2024-08-18T13:42:26.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:26.279+0000 7f39a759e700 1 -- 172.21.15.118:0/2494925701 wait complete. 2024-08-18T13:42:26.285 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 11 2024-08-18T13:42:26.295 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:28.296 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:28.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 Processor -- start 2024-08-18T13:42:28.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- start start 2024-08-18T13:42:28.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2b640c8200 0x7f2b640673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:28.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6406ee60 con 0x7f2b6406f2f0 2024-08-18T13:42:28.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6406f730 con 0x7f2b64153690 2024-08-18T13:42:28.415 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b6415a640 con 0x7f2b640c8200 2024-08-18T13:42:28.415 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b63fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2b640c8200 0x7f2b640673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:28.415 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b691f7700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2b6406f2f0 0x7f2b640c29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36668/0 (socket says 172.21.15.118:36668) 2024-08-18T13:42:28.415 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b63fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2b640c8200 0x7f2b640673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51642/0 (socket says 172.21.15.118:51642) 2024-08-18T13:42:28.415 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b691f7700 1 -- 172.21.15.118:0/2972763374 learned_addr learned my addr 172.21.15.118:0/2972763374 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:28.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b63fff700 1 -- 172.21.15.118:0/2972763374 >> v1:172.21.15.178:6789/0 conn(0x7f2b64153690 legacy=0x7f2b640c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:28.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b63fff700 1 -- 172.21.15.118:0/2972763374 >> v1:172.21.15.118:6789/0 conn(0x7f2b6406f2f0 legacy=0x7f2b640c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:28.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b63fff700 1 -- 172.21.15.118:0/2972763374 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b64153d10 con 0x7f2b640c8200 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b63fff700 1 --2- 172.21.15.118:0/2972763374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b640c8200 0x7f2b640673d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b580098e0 tx=0x7f2b580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f79a80b5a7230 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b637fe700 1 -- 172.21.15.118:0/2972763374 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58009e80 con 0x7f2b640c8200 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b637fe700 1 -- 172.21.15.118:0/2972763374 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5800b860 con 0x7f2b640c8200 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b637fe700 1 -- 172.21.15.118:0/2972763374 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800bbe0 con 0x7f2b640c8200 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- 172.21.15.118:0/2972763374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b640c8200 msgr2=0x7f2b640673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 --2- 172.21.15.118:0/2972763374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b640c8200 0x7f2b640673d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b580098e0 tx=0x7f2b580091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- 172.21.15.118:0/2972763374 shutdown_connections 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 --2- 172.21.15.118:0/2972763374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b640c8200 0x7f2b640673d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- 172.21.15.118:0/2972763374 >> 172.21.15.118:0/2972763374 conn(0x7f2b6406c020 msgr2=0x7f2b640c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- 172.21.15.118:0/2972763374 shutdown_connections 2024-08-18T13:42:28.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.411+0000 7f2b699f8700 1 -- 172.21.15.118:0/2972763374 wait complete. 2024-08-18T13:42:28.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 Processor -- start 2024-08-18T13:42:28.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 -- start start 2024-08-18T13:42:28.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b6406f2f0 0x7f2b6414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:28.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b640c8200 0x7f2b6414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b64153690 0x7f2b640befb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6414fee0 con 0x7f2b64153690 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6416a510 con 0x7f2b6406f2f0 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64154620 con 0x7f2b640c8200 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b64153690 0x7f2b640befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b64153690 0x7f2b640befb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51652/0 (socket says 172.21.15.118:51652) 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b689f6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b6406f2f0 0x7f2b6414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 -- 172.21.15.118:0/3697501711 learned_addr learned my addr 172.21.15.118:0/3697501711 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b63fff700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b640c8200 0x7f2b6414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 -- 172.21.15.118:0/3697501711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b640c8200 msgr2=0x7f2b6414dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b640c8200 0x7f2b6414dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 -- 172.21.15.118:0/3697501711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b6406f2f0 msgr2=0x7f2b6414f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b6406f2f0 0x7f2b6414f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 -- 172.21.15.118:0/3697501711 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009420 con 0x7f2b64153690 2024-08-18T13:42:28.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b691f7700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b64153690 0x7f2b640befb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c5c0 tx=0x7f2b5c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:28.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b60ff9700 1 -- 172.21.15.118:0/3697501711 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c005b30 con 0x7f2b64153690 2024-08-18T13:42:28.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b60ff9700 1 -- 172.21.15.118:0/3697501711 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5c005cb0 con 0x7f2b64153690 2024-08-18T13:42:28.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b60ff9700 1 -- 172.21.15.118:0/3697501711 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c007320 con 0x7f2b64153690 2024-08-18T13:42:28.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6414e290 con 0x7f2b64153690 2024-08-18T13:42:28.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.415+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b64169be0 con 0x7f2b64153690 2024-08-18T13:42:28.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.419+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b640713a0 con 0x7f2b64153690 2024-08-18T13:42:28.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.419+0000 7f2b60ff9700 1 -- 172.21.15.118:0/3697501711 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2b5c01b3b0 con 0x7f2b64153690 2024-08-18T13:42:28.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.419+0000 7f2b60ff9700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2b50060890 0x7f2b50062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:28.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.419+0000 7f2b60ff9700 1 -- 172.21.15.118:0/3697501711 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f2b5c07c2d0 con 0x7f2b64153690 2024-08-18T13:42:28.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.419+0000 7f2b689f6700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2b50060890 0x7f2b50062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:28.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.419+0000 7f2b689f6700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2b50060890 0x7f2b50062d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b54007fd0 tx=0x7f2b54001630 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:28.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.423+0000 7f2b60ff9700 1 -- 172.21.15.118:0/3697501711 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2b5c04c310 con 0x7f2b64153690 2024-08-18T13:42:28.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.623+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b6416a650 con 0x7f2b64153690 2024-08-18T13:42:28.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.623+0000 7f2b60ff9700 1 -- 172.21.15.118:0/3697501711 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f2b5c04f930 con 0x7f2b64153690 2024-08-18T13:42:28.627 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:28.627 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:27.638688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:rejoin","state_seq":47,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:28.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2b50060890 msgr2=0x7f2b50062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:28.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2b50060890 0x7f2b50062d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b54007fd0 tx=0x7f2b54001630 comp rx=0 tx=0).stop 2024-08-18T13:42:28.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b64153690 msgr2=0x7f2b640befb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:28.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b64153690 0x7f2b640befb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c5c0 tx=0x7f2b5c007ef0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 shutdown_connections 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2b50060890 0x7f2b50062d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b64153690 0x7f2b640befb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b640c8200 0x7f2b6414dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 --2- 172.21.15.118:0/3697501711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b6406f2f0 0x7f2b6414f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 >> 172.21.15.118:0/3697501711 conn(0x7f2b6406c020 msgr2=0x7f2b640c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 shutdown_connections 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:28.627+0000 7f2b699f8700 1 -- 172.21.15.118:0/3697501711 wait complete. 2024-08-18T13:42:28.630 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 12 2024-08-18T13:42:28.640 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 12, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 12, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4306": { "addr": "172.21.15.178:6835/1590569622", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6832", "nonce": 1590569622, "type": "v2" }, { "addr": "172.21.15.178:6835", "nonce": 1590569622, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4306, "incarnation": 10, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:rejoin", "state_seq": 47 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:42:27.638688+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4306 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:42:30.641 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:30.764 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.759+0000 7f329bfff700 1 Processor -- start 2024-08-18T13:42:30.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.759+0000 7f329bfff700 1 -- start start 2024-08-18T13:42:30.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329bfff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 0x7f329c0c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f329c06ba70 0x7f329c06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.768 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329bfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329c0ca970 con 0x7f329c156be0 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329bfff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c06c400 con 0x7f329c06ba70 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329bfff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0becb0 con 0x7f329c156810 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329b7fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f329c156be0 0x7f329c157060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36670/0 (socket says 172.21.15.118:36670) 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329b7fe700 1 -- 172.21.15.118:0/4214351543 learned_addr learned my addr 172.21.15.118:0/4214351543 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 --2- 172.21.15.118:0/4214351543 >> v2:172.21.15.178:3300/0 conn(0x7f329c06ba70 0x7f329c06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329affd700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 0x7f329c0c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009117059 0 0) 0x7f329c0ca970 con 0x7f329c156be0 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3288003570 con 0x7f329c156be0 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358336164 0 0) 0x7f3288003570 con 0x7f329c156be0 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 msgr2=0x7f329c0c07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 0x7f329c0c07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 >> v2:172.21.15.178:3300/0 conn(0x7f329c06ba70 msgr2=0x7f329c06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:30.769 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 --2- 172.21.15.118:0/4214351543 >> v2:172.21.15.178:3300/0 conn(0x7f329c06ba70 0x7f329c06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c0bef90 con 0x7f329c156be0 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f32940017f0 con 0x7f329c156be0 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 >> v1:172.21.15.118:6789/0 conn(0x7f329c156be0 legacy=0x7f329c157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3288003a60 0x7f3288003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f3288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 0x7f32880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c06c400 con 0x7f3288003a60 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0becb0 con 0x7f3288004810 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328800ba60 con 0x7f329c156810 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329affd700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3288003a60 0x7f3288003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f3288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329b7fe700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 0x7f32880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.770 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 -- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 msgr2=0x7f32880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:30.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c156810 0x7f32880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 -- 172.21.15.118:0/4214351543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3288003a60 msgr2=0x7f3288003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:30.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3288003a60 0x7f3288003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 -- 172.21.15.118:0/4214351543 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288003570 con 0x7f3288004810 2024-08-18T13:42:30.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329affd700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3288003a60 0x7f3288003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:42:30.771 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.763+0000 7f329a7fc700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f3288004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f328c009c30 tx=0x7f328c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f51478d851d6f5c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:30.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c00e710 con 0x7f3288004810 2024-08-18T13:42:30.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c00e870 con 0x7f3288004810 2024-08-18T13:42:30.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f3299ffb700 1 -- 172.21.15.118:0/4214351543 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c00ec30 con 0x7f3288004810 2024-08-18T13:42:30.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 -- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 msgr2=0x7f3288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:30.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 --2- 172.21.15.118:0/4214351543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f3288004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f328c009c30 tx=0x7f328c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.772 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329b7fe700 1 -- 172.21.15.118:0/4214351543 reap_dead start 2024-08-18T13:42:30.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 -- 172.21.15.118:0/4214351543 shutdown_connections 2024-08-18T13:42:30.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 -- 172.21.15.118:0/4214351543 >> 172.21.15.118:0/4214351543 conn(0x7f329c06e090 msgr2=0x7f329c0c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:30.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 -- 172.21.15.118:0/4214351543 shutdown_connections 2024-08-18T13:42:30.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 -- 172.21.15.118:0/4214351543 wait complete. 2024-08-18T13:42:30.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 Processor -- start 2024-08-18T13:42:30.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.767+0000 7f329bfff700 1 -- start start 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f329c162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c0648f0 0x7f329c1486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f329c0c9040 0x7f329c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c163310 con 0x7f329c0c9040 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c151100 con 0x7f3288004810 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c151c80 con 0x7f329c0648f0 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f329c0c9040 0x7f329c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f329c0c9040 0x7f329c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51684/0 (socket says 172.21.15.118:51684) 2024-08-18T13:42:30.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 -- 172.21.15.118:0/1124308383 learned_addr learned my addr 172.21.15.118:0/1124308383 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:30.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329a7fc700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c0648f0 0x7f329c1486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329affd700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f329c162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 -- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c0648f0 msgr2=0x7f329c1486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c0648f0 0x7f329c1486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 -- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 msgr2=0x7f329c162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f329c162dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 -- 172.21.15.118:0/1124308383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c0c4c30 con 0x7f329c0c9040 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329b7fe700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f329c0c9040 0x7f329c0c94c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f329c06f8a0 tx=0x7f32940021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f3292ffd700 1 -- 172.21.15.118:0/1124308383 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400ad30 con 0x7f329c0c9040 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f3292ffd700 1 -- 172.21.15.118:0/1124308383 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32940133f0 con 0x7f329c0c9040 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c151350 con 0x7f329c0c9040 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f3292ffd700 1 -- 172.21.15.118:0/1124308383 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32940137b0 con 0x7f329c0c9040 2024-08-18T13:42:30.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c151910 con 0x7f329c0c9040 2024-08-18T13:42:30.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.771+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c070620 con 0x7f329c0c9040 2024-08-18T13:42:30.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.775+0000 7f3292ffd700 1 -- 172.21.15.118:0/1124308383 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3294013ba0 con 0x7f329c0c9040 2024-08-18T13:42:30.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.775+0000 7f3292ffd700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f327c060860 0x7f327c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:30.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.775+0000 7f3292ffd700 1 -- 172.21.15.118:0/1124308383 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f329407d8f0 con 0x7f329c0c9040 2024-08-18T13:42:30.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.775+0000 7f329affd700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f327c060860 0x7f327c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:30.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.775+0000 7f329affd700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f327c060860 0x7f327c062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3284006f00 tx=0x7f32840091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:30.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.783+0000 7f3292ffd700 1 -- 172.21.15.118:0/1124308383 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f329404d930 con 0x7f329c0c9040 2024-08-18T13:42:30.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.987+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f329c0713a0 con 0x7f329c0c9040 2024-08-18T13:42:30.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.987+0000 7f3292ffd700 1 -- 172.21.15.118:0/1124308383 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f3294050f50 con 0x7f329c0c9040 2024-08-18T13:42:30.992 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:30.992 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:27.638688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:rejoin","state_seq":47,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f327c060860 msgr2=0x7f327c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f327c060860 0x7f327c062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3284006f00 tx=0x7f32840091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f329c0c9040 msgr2=0x7f329c0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f329c0c9040 0x7f329c0c94c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f329c06f8a0 tx=0x7f32940021d0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 shutdown_connections 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f327c060860 0x7f327c062d10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f329c0c9040 0x7f329c0c94c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f329c0648f0 0x7f329c1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 --2- 172.21.15.118:0/1124308383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3288004810 0x7f329c162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:30.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 >> 172.21.15.118:0/1124308383 conn(0x7f329c06e090 msgr2=0x7f329c14b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:30.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 shutdown_connections 2024-08-18T13:42:30.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:30.991+0000 7f329bfff700 1 -- 172.21.15.118:0/1124308383 wait complete. 2024-08-18T13:42:30.996 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 12 2024-08-18T13:42:31.006 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:33.007 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:33.122 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17dc89700 1 Processor -- start 2024-08-18T13:42:33.122 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17dc89700 1 -- start start 2024-08-18T13:42:33.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17dc89700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1780c4320 0x7fa178156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17dc89700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa178067d50 0x7fa178153980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17dc89700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1781534f0 con 0x7fa1780c46f0 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17dc89700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1781574c0 con 0x7fa178067d50 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17dc89700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa178152f70 con 0x7fa1780c4320 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17d488700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa178067d50 0x7fa178153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa177fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fa1780c46f0 0x7fa17805e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36682/0 (socket says 172.21.15.118:36682) 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa177fff700 1 -- 172.21.15.118:0/1734710576 learned_addr learned my addr 172.21.15.118:0/1734710576 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17d488700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa178067d50 0x7fa178153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:51692/0 (socket says 172.21.15.118:51692) 2024-08-18T13:42:33.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.119+0000 7fa17cc87700 1 --2- 172.21.15.118:0/1734710576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1780c4320 0x7fa178156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:33.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17cc87700 1 -- 172.21.15.118:0/1734710576 >> v2:172.21.15.118:3300/0 conn(0x7fa178067d50 msgr2=0x7fa178153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:33.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17cc87700 1 --2- 172.21.15.118:0/1734710576 >> v2:172.21.15.118:3300/0 conn(0x7fa178067d50 0x7fa178153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17cc87700 1 -- 172.21.15.118:0/1734710576 >> v1:172.21.15.118:6789/0 conn(0x7fa1780c46f0 legacy=0x7fa17805e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:33.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17cc87700 1 -- 172.21.15.118:0/1734710576 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178156c20 con 0x7fa1780c4320 2024-08-18T13:42:33.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17cc87700 1 --2- 172.21.15.118:0/1734710576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1780c4320 0x7fa178156570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6a6e310127b07c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:33.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa1777fe700 1 -- 172.21.15.118:0/1734710576 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00b640 con 0x7fa1780c4320 2024-08-18T13:42:33.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa1777fe700 1 -- 172.21.15.118:0/1734710576 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c00b7a0 con 0x7fa1780c4320 2024-08-18T13:42:33.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa1777fe700 1 -- 172.21.15.118:0/1734710576 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00bb40 con 0x7fa1780c4320 2024-08-18T13:42:33.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 -- 172.21.15.118:0/1734710576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1780c4320 msgr2=0x7fa178156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:33.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 --2- 172.21.15.118:0/1734710576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1780c4320 0x7fa178156570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0098e0 tx=0x7fa16c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 -- 172.21.15.118:0/1734710576 shutdown_connections 2024-08-18T13:42:33.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 --2- 172.21.15.118:0/1734710576 >> v2:172.21.15.118:3300/0 conn(0x7fa178067d50 0x7fa178153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 --2- 172.21.15.118:0/1734710576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1780c4320 0x7fa178156570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 -- 172.21.15.118:0/1734710576 >> 172.21.15.118:0/1734710576 conn(0x7fa1780c81e0 msgr2=0x7fa1780c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:33.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 -- 172.21.15.118:0/1734710576 shutdown_connections 2024-08-18T13:42:33.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.123+0000 7fa17dc89700 1 -- 172.21.15.118:0/1734710576 wait complete. 2024-08-18T13:42:33.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 Processor -- start 2024-08-18T13:42:33.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 -- start start 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa178067d50 0x7fa178141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1780c4320 0x7fa1780c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1780c46f0 0x7fa1780c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa178152c70 con 0x7fa1780c46f0 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780c69e0 con 0x7fa1780c4320 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17dc89700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781418a0 con 0x7fa178067d50 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17d488700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1780c46f0 0x7fa1780c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17cc87700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa178067d50 0x7fa178141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:33.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17cc87700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa178067d50 0x7fa178141360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:47348/0 (socket says 172.21.15.118:47348) 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.127+0000 7fa17cc87700 1 -- 172.21.15.118:0/3064020456 learned_addr learned my addr 172.21.15.118:0/3064020456 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa177fff700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1780c4320 0x7fa1780c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17d488700 1 -- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa178067d50 msgr2=0x7fa178141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17d488700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa178067d50 0x7fa178141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17d488700 1 -- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1780c4320 msgr2=0x7fa1780c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17d488700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1780c4320 0x7fa1780c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17d488700 1 -- 172.21.15.118:0/3064020456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c009420 con 0x7fa1780c46f0 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa177fff700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1780c4320 0x7fa1780c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17d488700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1780c46f0 0x7fa1780c73b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa170009e80 tx=0x7fa17000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa174ff9700 1 -- 172.21.15.118:0/3064020456 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000cb50 con 0x7fa1780c46f0 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa174ff9700 1 -- 172.21.15.118:0/3064020456 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17000f040 con 0x7fa1780c46f0 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1781464c0 con 0x7fa1780c46f0 2024-08-18T13:42:33.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa174ff9700 1 -- 172.21.15.118:0/3064020456 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170014480 con 0x7fa1780c46f0 2024-08-18T13:42:33.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1780c6390 con 0x7fa1780c46f0 2024-08-18T13:42:33.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1780c61f0 con 0x7fa1780c46f0 2024-08-18T13:42:33.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa174ff9700 1 -- 172.21.15.118:0/3064020456 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa1700147f0 con 0x7fa1780c46f0 2024-08-18T13:42:33.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa174ff9700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa15c060860 0x7fa15c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:33.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.131+0000 7fa174ff9700 1 -- 172.21.15.118:0/3064020456 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7fa17007dbe0 con 0x7fa1780c46f0 2024-08-18T13:42:33.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.135+0000 7fa17cc87700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa15c060860 0x7fa15c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:33.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.135+0000 7fa17cc87700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa15c060860 0x7fa15c062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c010 tx=0x7fa16c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:33.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.139+0000 7fa174ff9700 1 -- 172.21.15.118:0/3064020456 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa1780c61f0 con 0x7fa1780c46f0 2024-08-18T13:42:33.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.343+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1780c61f0 con 0x7fa1780c46f0 2024-08-18T13:42:33.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.343+0000 7fa174ff9700 1 -- 172.21.15.118:0/3064020456 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fa1780c61f0 con 0x7fa1780c46f0 2024-08-18T13:42:33.347 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:33.347 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:27.638688+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:rejoin","state_seq":47,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:33.349 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa15c060860 msgr2=0x7fa15c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa15c060860 0x7fa15c062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c010 tx=0x7fa16c017000 comp rx=0 tx=0).stop 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1780c46f0 msgr2=0x7fa1780c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1780c46f0 0x7fa1780c73b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa170009e80 tx=0x7fa17000b040 comp rx=0 tx=0).stop 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 shutdown_connections 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa15c060860 0x7fa15c062d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1780c46f0 0x7fa1780c73b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1780c4320 0x7fa1780c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 --2- 172.21.15.118:0/3064020456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa178067d50 0x7fa178141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 >> 172.21.15.118:0/3064020456 conn(0x7fa1780c81e0 msgr2=0x7fa17806af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 shutdown_connections 2024-08-18T13:42:33.350 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:33.347+0000 7fa17dc89700 1 -- 172.21.15.118:0/3064020456 wait complete. 2024-08-18T13:42:33.351 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 12 2024-08-18T13:42:33.360 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:35.362 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:35.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a9b6d700 1 Processor -- start 2024-08-18T13:42:35.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a9b6d700 1 -- start start 2024-08-18T13:42:35.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a9b6d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f17a414bf60 0x7f17a4152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:35.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a9b6d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:35.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a9b6d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a4067470 con 0x7f17a406c1e0 2024-08-18T13:42:35.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a9b6d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a4068370 con 0x7f17a414bf60 2024-08-18T13:42:35.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a9b6d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4152d80 con 0x7f17a405e6b0 2024-08-18T13:42:35.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a8b6b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f17a414bf60 0x7f17a4152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a936c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f17a406c1e0 0x7f17a40c0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55654/0 (socket says 172.21.15.118:55654) 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a8b6b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f17a414bf60 0x7f17a4152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53750/0 (socket says 172.21.15.118:53750) 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a936c700 1 -- 172.21.15.118:0/943638478 learned_addr learned my addr 172.21.15.118:0/943638478 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a3fff700 1 --2- 172.21.15.118:0/943638478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a8b6b700 1 -- 172.21.15.118:0/943638478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 msgr2=0x7f17a406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a8b6b700 1 --2- 172.21.15.118:0/943638478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a406bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a8b6b700 1 -- 172.21.15.118:0/943638478 >> v1:172.21.15.118:6789/0 conn(0x7f17a406c1e0 legacy=0x7f17a40c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a8b6b700 1 -- 172.21.15.118:0/943638478 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4067710 con 0x7f17a414bf60 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.467+0000 7f17a3fff700 1 --2- 172.21.15.118:0/943638478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a406bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a8b6b700 1 --2- 172.21.15.118:0/943638478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a414bf60 0x7f17a4152840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17940098e0 tx=0x7f17940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d9c65a6f740902 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a37fe700 1 -- 172.21.15.118:0/943638478 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794009e80 con 0x7f17a414bf60 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a37fe700 1 -- 172.21.15.118:0/943638478 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179400b8f0 con 0x7f17a414bf60 2024-08-18T13:42:35.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a37fe700 1 -- 172.21.15.118:0/943638478 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179400bc70 con 0x7f17a414bf60 2024-08-18T13:42:35.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- 172.21.15.118:0/943638478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a414bf60 msgr2=0x7f17a4152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/943638478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a414bf60 0x7f17a4152840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17940098e0 tx=0x7f17940091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- 172.21.15.118:0/943638478 shutdown_connections 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/943638478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a406bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/943638478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a414bf60 0x7f17a4152840 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- 172.21.15.118:0/943638478 >> 172.21.15.118:0/943638478 conn(0x7f17a40c81e0 msgr2=0x7f17a40c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- 172.21.15.118:0/943638478 shutdown_connections 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- 172.21.15.118:0/943638478 wait complete. 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 Processor -- start 2024-08-18T13:42:35.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- start start 2024-08-18T13:42:35.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a4161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:35.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a406c1e0 0x7f17a414eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:35.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a414bf60 0x7f17a414d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:35.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40b4ba0 con 0x7f17a406c1e0 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a414e660 con 0x7f17a414bf60 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a9b6d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4161cd0 con 0x7f17a405e6b0 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a406c1e0 0x7f17a414eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a406c1e0 0x7f17a414eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53766/0 (socket says 172.21.15.118:53766) 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.471+0000 7f17a3fff700 1 -- 172.21.15.118:0/3064017136 learned_addr learned my addr 172.21.15.118:0/3064017136 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a936c700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a414bf60 0x7f17a414d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a8b6b700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a4161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a3fff700 1 -- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 msgr2=0x7f17a4161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a3fff700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a4161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a3fff700 1 -- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a414bf60 msgr2=0x7f17a414d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a3fff700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a414bf60 0x7f17a414d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a3fff700 1 -- 172.21.15.118:0/3064017136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794009420 con 0x7f17a406c1e0 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a936c700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a414bf60 0x7f17a414d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:35.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a3fff700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a406c1e0 0x7f17a414eaf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f179800e900 tx=0x7f1798008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:35.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a0ff9700 1 -- 172.21.15.118:0/3064017136 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798013410 con 0x7f17a406c1e0 2024-08-18T13:42:35.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a0ff9700 1 -- 172.21.15.118:0/3064017136 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1798013570 con 0x7f17a406c1e0 2024-08-18T13:42:35.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a40bf0e0 con 0x7f17a406c1e0 2024-08-18T13:42:35.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a0ff9700 1 -- 172.21.15.118:0/3064017136 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798013910 con 0x7f17a406c1e0 2024-08-18T13:42:35.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a414e010 con 0x7f17a406c1e0 2024-08-18T13:42:35.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a40c0400 con 0x7f17a406c1e0 2024-08-18T13:42:35.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.475+0000 7f17a0ff9700 1 -- 172.21.15.118:0/3064017136 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1798013d00 con 0x7f17a406c1e0 2024-08-18T13:42:35.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.479+0000 7f17a0ff9700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1790060890 0x7f1790062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:35.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.479+0000 7f17a8b6b700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1790060890 0x7f1790062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:35.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.479+0000 7f17a0ff9700 1 -- 172.21.15.118:0/3064017136 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f179807cc60 con 0x7f17a406c1e0 2024-08-18T13:42:35.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.479+0000 7f17a8b6b700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1790060890 0x7f1790062d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f17940091e0 tx=0x7f1794017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:35.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.483+0000 7f17a0ff9700 1 -- 172.21.15.118:0/3064017136 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f17a40c0400 con 0x7f17a406c1e0 2024-08-18T13:42:35.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.687+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17a40c0400 con 0x7f17a406c1e0 2024-08-18T13:42:35.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.687+0000 7f17a0ff9700 1 -- 172.21.15.118:0/3064017136 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5376 (secure 0 0 0) 0x7f17a40c0400 con 0x7f17a406c1e0 2024-08-18T13:42:35.690 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:35.691 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:33.664260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:clientreplay","state_seq":49,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:35.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.687+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1790060890 msgr2=0x7f1790062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:35.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.687+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1790060890 0x7f1790062d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f17940091e0 tx=0x7f1794017000 comp rx=0 tx=0).stop 2024-08-18T13:42:35.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.687+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a406c1e0 msgr2=0x7f17a414eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:35.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.687+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a406c1e0 0x7f17a414eaf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f179800e900 tx=0x7f1798008040 comp rx=0 tx=0).stop 2024-08-18T13:42:35.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 shutdown_connections 2024-08-18T13:42:35.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1790060890 0x7f1790062d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17a414bf60 0x7f17a414d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a406c1e0 0x7f17a414eaf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 --2- 172.21.15.118:0/3064017136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17a405e6b0 0x7f17a4161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:35.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 >> 172.21.15.118:0/3064017136 conn(0x7f17a40c81e0 msgr2=0x7f17a40c2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:35.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 shutdown_connections 2024-08-18T13:42:35.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:35.691+0000 7f17a9b6d700 1 -- 172.21.15.118:0/3064017136 wait complete. 2024-08-18T13:42:35.694 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 13 2024-08-18T13:42:35.704 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:37.705 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:37.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad3abd6700 1 Processor -- start 2024-08-18T13:42:37.822 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad3abd6700 1 -- start start 2024-08-18T13:42:37.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad3abd6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fad340c27e0 0x7fad340c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:37.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad3abd6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fad3414f1e0 0x7fad3414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:37.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad3abd6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340c0af0 con 0x7fad340c76d0 2024-08-18T13:42:37.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad3abd6700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340ac450 con 0x7fad340c27e0 2024-08-18T13:42:37.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad3abd6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad34156040 con 0x7fad3414f1e0 2024-08-18T13:42:37.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad33fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fad340c27e0 0x7fad340c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:37.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad38972700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fad340c76d0 0x7fad340ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55664/0 (socket says 172.21.15.118:55664) 2024-08-18T13:42:37.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad38972700 1 -- 172.21.15.118:0/726979158 learned_addr learned my addr 172.21.15.118:0/726979158 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:37.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad39173700 1 --2- 172.21.15.118:0/726979158 >> v2:172.21.15.178:3300/0 conn(0x7fad3414f1e0 0x7fad3414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad33fff700 1 -- 172.21.15.118:0/726979158 >> v2:172.21.15.178:3300/0 conn(0x7fad3414f1e0 msgr2=0x7fad3414f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad33fff700 1 --2- 172.21.15.118:0/726979158 >> v2:172.21.15.178:3300/0 conn(0x7fad3414f1e0 0x7fad3414f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad33fff700 1 -- 172.21.15.118:0/726979158 >> v1:172.21.15.118:6789/0 conn(0x7fad340c76d0 legacy=0x7fad340ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad33fff700 1 -- 172.21.15.118:0/726979158 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3414fb20 con 0x7fad340c27e0 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad33fff700 1 --2- 172.21.15.118:0/726979158 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c27e0 0x7fad340c7aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fad3406a6a0 tx=0x7fad1c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258fb4bae506573c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad337fe700 1 -- 172.21.15.118:0/726979158 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00e070 con 0x7fad340c27e0 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad337fe700 1 -- 172.21.15.118:0/726979158 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c00bbf0 con 0x7fad340c27e0 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.819+0000 7fad337fe700 1 -- 172.21.15.118:0/726979158 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c012430 con 0x7fad340c27e0 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- 172.21.15.118:0/726979158 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c27e0 msgr2=0x7fad340c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:37.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 --2- 172.21.15.118:0/726979158 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c27e0 0x7fad340c7aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fad3406a6a0 tx=0x7fad1c009780 comp rx=0 tx=0).stop 2024-08-18T13:42:37.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- 172.21.15.118:0/726979158 shutdown_connections 2024-08-18T13:42:37.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 --2- 172.21.15.118:0/726979158 >> v2:172.21.15.178:3300/0 conn(0x7fad3414f1e0 0x7fad3414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:37.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 --2- 172.21.15.118:0/726979158 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c27e0 0x7fad340c7aa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:37.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- 172.21.15.118:0/726979158 >> 172.21.15.118:0/726979158 conn(0x7fad340cc800 msgr2=0x7fad340c87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:37.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- 172.21.15.118:0/726979158 shutdown_connections 2024-08-18T13:42:37.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- 172.21.15.118:0/726979158 wait complete. 2024-08-18T13:42:37.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 Processor -- start 2024-08-18T13:42:37.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- start start 2024-08-18T13:42:37.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad340c27e0 0x7fad34169370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c76d0 0x7fad341589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad3414f1e0 0x7fad341545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34157910 con 0x7fad340c76d0 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34154170 con 0x7fad3414f1e0 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad3abd6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad34156580 con 0x7fad340c27e0 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c76d0 0x7fad341589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c76d0 0x7fad341589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53776/0 (socket says 172.21.15.118:53776) 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 -- 172.21.15.118:0/1284947714 learned_addr learned my addr 172.21.15.118:0/1284947714 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad38972700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad340c27e0 0x7fad34169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad39173700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad3414f1e0 0x7fad341545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 -- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad340c27e0 msgr2=0x7fad34169370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad340c27e0 0x7fad34169370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:37.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 -- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad3414f1e0 msgr2=0x7fad341545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad3414f1e0 0x7fad341545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad33fff700 1 -- 172.21.15.118:0/1284947714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c009a20 con 0x7fad340c76d0 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad38972700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad340c27e0 0x7fad34169370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.823+0000 7fad39173700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad3414f1e0 0x7fad341545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad33fff700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c76d0 0x7fad341589d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0097c0 tx=0x7fad1c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad30ff9700 1 -- 172.21.15.118:0/1284947714 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c012e60 con 0x7fad340c76d0 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad30ff9700 1 -- 172.21.15.118:0/1284947714 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c027470 con 0x7fad340c76d0 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad340c7fe0 con 0x7fad340c76d0 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad30ff9700 1 -- 172.21.15.118:0/1284947714 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c027830 con 0x7fad340c76d0 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad34156a50 con 0x7fad340c76d0 2024-08-18T13:42:37.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad340c9740 con 0x7fad340c76d0 2024-08-18T13:42:37.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad30ff9700 1 -- 172.21.15.118:0/1284947714 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fad1c024070 con 0x7fad340c76d0 2024-08-18T13:42:37.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad30ff9700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fad18060860 0x7fad18062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:37.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad30ff9700 1 -- 172.21.15.118:0/1284947714 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7fad1c07f620 con 0x7fad340c76d0 2024-08-18T13:42:37.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.827+0000 7fad38972700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fad18060860 0x7fad18062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:37.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.831+0000 7fad38972700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fad18060860 0x7fad18062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad240018c0 tx=0x7fad24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:37.838 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:37.835+0000 7fad30ff9700 1 -- 172.21.15.118:0/1284947714 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fad340c9740 con 0x7fad340c76d0 2024-08-18T13:42:38.040 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.035+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad340c9740 con 0x7fad340c76d0 2024-08-18T13:42:38.041 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.035+0000 7fad30ff9700 1 -- 172.21.15.118:0/1284947714 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5376 (secure 0 0 0) 0x7fad340c9740 con 0x7fad340c76d0 2024-08-18T13:42:38.041 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:38.041 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:33.664260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:clientreplay","state_seq":49,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:38.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fad18060860 msgr2=0x7fad18062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:38.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fad18060860 0x7fad18062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad240018c0 tx=0x7fad24009040 comp rx=0 tx=0).stop 2024-08-18T13:42:38.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c76d0 msgr2=0x7fad341589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:38.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c76d0 0x7fad341589d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0097c0 tx=0x7fad1c019000 comp rx=0 tx=0).stop 2024-08-18T13:42:38.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 shutdown_connections 2024-08-18T13:42:38.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fad18060860 0x7fad18062d10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:38.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad3414f1e0 0x7fad341545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:38.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad340c76d0 0x7fad341589d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:38.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 --2- 172.21.15.118:0/1284947714 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad340c27e0 0x7fad34169370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:38.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 >> 172.21.15.118:0/1284947714 conn(0x7fad340cc800 msgr2=0x7fad340c0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:38.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 shutdown_connections 2024-08-18T13:42:38.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:38.039+0000 7fad3abd6700 1 -- 172.21.15.118:0/1284947714 wait complete. 2024-08-18T13:42:38.044 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 13 2024-08-18T13:42:38.054 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:42:40.055 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:40.174 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 Processor -- start 2024-08-18T13:42:40.174 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 -- start start 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffa9c14bf60 0x7ffa9c152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffa9c05e6b0 0x7ffa9c06bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c06c1e0 0x7ffa9c0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c068370 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c152d80 con 0x7ffa9c14bf60 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa34c7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c067470 con 0x7ffa9c06c1e0 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffa9c05e6b0 0x7ffa9c06bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.176 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ffa9c05e6b0 0x7ffa9c06bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53790/0 (socket says 172.21.15.118:53790) 2024-08-18T13:42:40.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 -- 172.21.15.118:0/807914680 learned_addr learned my addr 172.21.15.118:0/807914680 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:40.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa24c5700 1 --2- 172.21.15.118:0/807914680 >> v2:172.21.15.178:3300/0 conn(0x7ffa9c14bf60 0x7ffa9c152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa2cc6700 1 --2- 172.21.15.118:0/807914680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c06c1e0 0x7ffa9c0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 -- 172.21.15.118:0/807914680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c06c1e0 msgr2=0x7ffa9c0c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 --2- 172.21.15.118:0/807914680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c06c1e0 0x7ffa9c0c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 -- 172.21.15.118:0/807914680 >> v2:172.21.15.178:3300/0 conn(0x7ffa9c14bf60 msgr2=0x7ffa9c152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 --2- 172.21.15.118:0/807914680 >> v2:172.21.15.178:3300/0 conn(0x7ffa9c14bf60 0x7ffa9c152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa1cc4700 1 -- 172.21.15.118:0/807914680 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c067710 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa24c5700 1 --2- 172.21.15.118:0/807914680 >> v2:172.21.15.178:3300/0 conn(0x7ffa9c14bf60 0x7ffa9c152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.171+0000 7ffaa2cc6700 1 --2- 172.21.15.118:0/807914680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c06c1e0 0x7ffa9c0c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:42:40.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa1cc4700 1 --2- 172.21.15.118:0/807914680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c06bca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c06ec80 tx=0x7ffa8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd368ace1acfa110 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:40.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa14c3700 1 -- 172.21.15.118:0/807914680 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00b6d0 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa14c3700 1 -- 172.21.15.118:0/807914680 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c00b830 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa14c3700 1 -- 172.21.15.118:0/807914680 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00bbf0 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 -- 172.21.15.118:0/807914680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 msgr2=0x7ffa9c06bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/807914680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c06bca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c06ec80 tx=0x7ffa8c009250 comp rx=0 tx=0).stop 2024-08-18T13:42:40.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 -- 172.21.15.118:0/807914680 shutdown_connections 2024-08-18T13:42:40.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/807914680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c06c1e0 0x7ffa9c0c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/807914680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c06bca0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/807914680 >> v2:172.21.15.178:3300/0 conn(0x7ffa9c14bf60 0x7ffa9c152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 -- 172.21.15.118:0/807914680 >> 172.21.15.118:0/807914680 conn(0x7ffa9c0c81e0 msgr2=0x7ffa9c0c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:40.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 -- 172.21.15.118:0/807914680 shutdown_connections 2024-08-18T13:42:40.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.175+0000 7ffaa34c7700 1 -- 172.21.15.118:0/807914680 wait complete. 2024-08-18T13:42:40.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 Processor -- start 2024-08-18T13:42:40.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 -- start start 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa9c06c1e0 0x7ffa9c162b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c14bf60 0x7ffa9c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c1621c0 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c165850 con 0x7ffa9c06c1e0 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c165b10 con 0x7ffa9c14bf60 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c162610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53796/0 (socket says 172.21.15.118:53796) 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 -- 172.21.15.118:0/3909883116 learned_addr learned my addr 172.21.15.118:0/3909883116 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa1cc4700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa9c06c1e0 0x7ffa9c162b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa2cc6700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c14bf60 0x7ffa9c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 -- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c14bf60 msgr2=0x7ffa9c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c14bf60 0x7ffa9c152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 -- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa9c06c1e0 msgr2=0x7ffa9c162b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa9c06c1e0 0x7ffa9c162b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 -- 172.21.15.118:0/3909883116 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94009a20 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa2cc6700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c14bf60 0x7ffa9c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:40.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa24c5700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c162610 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800ec80 tx=0x7ffa980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:40.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffa927fc700 1 -- 172.21.15.118:0/3909883116 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800c850 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffa927fc700 1 -- 172.21.15.118:0/3909883116 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9800c9b0 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa98009420 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffa927fc700 1 -- 172.21.15.118:0/3909883116 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800cd30 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.179+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c0094f0 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.183+0000 7ffa927fc700 1 -- 172.21.15.118:0/3909883116 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ffa98013680 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.183+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c0713a0 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.183+0000 7ffa927fc700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ffa80060860 0x7ffa80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.183+0000 7ffa927fc700 1 -- 172.21.15.118:0/3909883116 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7ffa9807d290 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.183+0000 7ffaa1cc4700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ffa80060860 0x7ffa80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.183+0000 7ffaa1cc4700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ffa80060860 0x7ffa80062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c06ec80 tx=0x7ffa8c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:40.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.191+0000 7ffa927fc700 1 -- 172.21.15.118:0/3909883116 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ffa98050d60 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.395+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa9c14a240 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.395+0000 7ffa927fc700 1 -- 172.21.15.118:0/3909883116 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5370 (secure 0 0 0) 0x7ffa980508f0 con 0x7ffa9c05e6b0 2024-08-18T13:42:40.400 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:40.400 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:40.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ffa80060860 msgr2=0x7ffa80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ffa80060860 0x7ffa80062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c06ec80 tx=0x7ffa8c017000 comp rx=0 tx=0).stop 2024-08-18T13:42:40.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 msgr2=0x7ffa9c162610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c162610 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800ec80 tx=0x7ffa980091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 shutdown_connections 2024-08-18T13:42:40.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ffa80060860 0x7ffa80062d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa9c14bf60 0x7ffa9c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa9c06c1e0 0x7ffa9c162b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 --2- 172.21.15.118:0/3909883116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa9c05e6b0 0x7ffa9c162610 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 >> 172.21.15.118:0/3909883116 conn(0x7ffa9c0c81e0 msgr2=0x7ffa9c0c0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:40.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 shutdown_connections 2024-08-18T13:42:40.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.399+0000 7ffaa34c7700 1 -- 172.21.15.118:0/3909883116 wait complete. 2024-08-18T13:42:40.404 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 14 2024-08-18T13:42:40.414 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=4312 2024-08-18T13:42:40.414 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.d 2024-08-18T13:42:40.414 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2024-08-18T13:42:40.414 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-18T13:42:40.414 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-18T13:42:40.417 INFO:tasks.ceph.mds.d:Started 2024-08-18T13:42:40.417 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:42:40.497 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-08-18T13:42:40.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f1421d11700 1 Processor -- start 2024-08-18T13:42:40.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f1421d11700 1 -- start start 2024-08-18T13:42:40.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f1421d11700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f141c0bea90 0x7f141c14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f1421d11700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f1421d11700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c151f80 con 0x7f141c0bd580 2024-08-18T13:42:40.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f1421d11700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141c151990 con 0x7f141c0bea90 2024-08-18T13:42:40.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f1421d11700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0c2a50 con 0x7f141c06b620 2024-08-18T13:42:40.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f141b7fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f141c0bd580 0x7f141c152410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55668/0 (socket says 172.21.15.118:55668) 2024-08-18T13:42:40.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f141b7fe700 1 -- 172.21.15.118:0/2908746745 learned_addr learned my addr 172.21.15.118:0/2908746745 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:42:40.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f141affd700 1 --2- 172.21.15.118:0/2908746745 >> v2:172.21.15.178:3300/0 conn(0x7f141c0bea90 0x7f141c14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.523+0000 7f141bfff700 1 --2- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141a7fc700 1 -- 172.21.15.118:0/2908746745 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852544450 0 0) 0x7f141c151f80 con 0x7f141c0bd580 2024-08-18T13:42:40.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141a7fc700 1 -- 172.21.15.118:0/2908746745 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1404003570 con 0x7f141c0bd580 2024-08-18T13:42:40.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141affd700 1 -- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 msgr2=0x7f141c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:42:40.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141affd700 1 --2- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141affd700 1 -- 172.21.15.118:0/2908746745 >> v1:172.21.15.118:6789/0 conn(0x7f141c0bd580 legacy=0x7f141c152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141affd700 1 -- 172.21.15.118:0/2908746745 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0c3c40 con 0x7f141c0bea90 2024-08-18T13:42:40.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141affd700 1 --2- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c14bcd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14100098e0 tx=0x7f14100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ed0a5ad8d1e66c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:40.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141a7fc700 1 -- 172.21.15.118:0/2908746745 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410011630 con 0x7f141c0bea90 2024-08-18T13:42:40.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141a7fc700 1 -- 172.21.15.118:0/2908746745 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410011790 con 0x7f141c0bea90 2024-08-18T13:42:40.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f141a7fc700 1 -- 172.21.15.118:0/2908746745 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410011b10 con 0x7f141c0bea90 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 -- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 msgr2=0x7f141c14bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 --2- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c14bcd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14100098e0 tx=0x7f14100091a0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 -- 172.21.15.118:0/2908746745 shutdown_connections 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 --2- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 --2- 172.21.15.118:0/2908746745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c14bcd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 -- 172.21.15.118:0/2908746745 >> 172.21.15.118:0/2908746745 conn(0x7f141c0c7c40 msgr2=0x7f141c0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 -- 172.21.15.118:0/2908746745 shutdown_connections 2024-08-18T13:42:40.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.527+0000 7f1421d11700 1 -- 172.21.15.118:0/2908746745 wait complete. 2024-08-18T13:42:40.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 Processor -- start 2024-08-18T13:42:40.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 -- start start 2024-08-18T13:42:40.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c136520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f141c0bd580 0x7f141c1334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c133a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c140020 con 0x7f141c0bd580 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c14b2d0 con 0x7f141c0bea90 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1494f0 con 0x7f141c06b620 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f141c0bd580 0x7f141c1334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f141c0bd580 0x7f141c1334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53808/0 (socket says 172.21.15.118:53808) 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 -- 172.21.15.118:0/322642028 learned_addr learned my addr 172.21.15.118:0/322642028 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141bfff700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c133a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141b7fe700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 -- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 msgr2=0x7f141c136520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c136520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 -- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 msgr2=0x7f141c133a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c133a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 -- 172.21.15.118:0/322642028 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410009420 con 0x7f141c0bd580 2024-08-18T13:42:40.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141bfff700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c133a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:42:40.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f141affd700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f141c0bd580 0x7f141c1334f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1410009ed0 tx=0x7f1410017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:40.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f14037fe700 1 -- 172.21.15.118:0/322642028 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410011b10 con 0x7f141c0bd580 2024-08-18T13:42:40.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f14037fe700 1 -- 172.21.15.118:0/322642028 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141001a430 con 0x7f141c0bd580 2024-08-18T13:42:40.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c13fd40 con 0x7f141c0bd580 2024-08-18T13:42:40.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f14037fe700 1 -- 172.21.15.118:0/322642028 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141001a7d0 con 0x7f141c0bd580 2024-08-18T13:42:40.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.531+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c14af60 con 0x7f141c0bd580 2024-08-18T13:42:40.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.535+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c0642f0 con 0x7f141c0bd580 2024-08-18T13:42:40.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.535+0000 7f14037fe700 1 -- 172.21.15.118:0/322642028 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f141001ab80 con 0x7f141c0bd580 2024-08-18T13:42:40.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.535+0000 7f14037fe700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1408060890 0x7f1408062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:42:40.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.535+0000 7f14037fe700 1 -- 172.21.15.118:0/322642028 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f141007edd0 con 0x7f141c0bd580 2024-08-18T13:42:40.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.535+0000 7f141b7fe700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1408060890 0x7f1408062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:42:40.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.535+0000 7f141b7fe700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1408060890 0x7f1408062d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f140c0058d0 tx=0x7f140c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:42:40.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.547+0000 7f14037fe700 1 -- 172.21.15.118:0/322642028 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f141004ee10 con 0x7f141c0bd580 2024-08-18T13:42:40.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.747+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f141c13fb20 con 0x7f141c0bd580 2024-08-18T13:42:40.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f14037fe700 1 -- 172.21.15.118:0/322642028 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f1410015090 con 0x7f141c0bd580 2024-08-18T13:42:40.754 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:42:40.754 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:42:40.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1408060890 msgr2=0x7f1408062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1408060890 0x7f1408062d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f140c0058d0 tx=0x7f140c009040 comp rx=0 tx=0).stop 2024-08-18T13:42:40.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f141c0bd580 msgr2=0x7f141c1334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:42:40.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f141c0bd580 0x7f141c1334f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1410009ed0 tx=0x7f1410017000 comp rx=0 tx=0).stop 2024-08-18T13:42:40.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 shutdown_connections 2024-08-18T13:42:40.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1408060890 0x7f1408062d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f141c0bea90 0x7f141c133a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f141c0bd580 0x7f141c1334f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 --2- 172.21.15.118:0/322642028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f141c06b620 0x7f141c136520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:42:40.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 >> 172.21.15.118:0/322642028 conn(0x7f141c0c7c40 msgr2=0x7f141c140840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:42:40.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 shutdown_connections 2024-08-18T13:42:40.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:42:40.751+0000 7f1421d11700 1 -- 172.21.15.118:0/322642028 wait complete. 2024-08-18T13:42:40.757 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:42:40.768 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2024-08-18T13:42:40.768 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 81 secs before thrashing 2024-08-18T13:42:42.325 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:42:42.328 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:42:45.540 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2024-08-18T13:42:45.540 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2024-08-18T13:42:45.540 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2024-08-18T13:42:45.540 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2024-08-18T13:42:45.540 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2024-08-18T13:42:45.541 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2024-08-18T13:42:45.542 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2024-08-18T13:42:45.543 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2024-08-18T13:42:45.544 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2024-08-18T13:42:45.545 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2024-08-18T13:42:45.546 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2024-08-18T13:42:46.741 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2024-08-18T13:42:46.742 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2024-08-18T13:42:46.743 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2024-08-18T13:42:46.744 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2024-08-18T13:42:46.745 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2024-08-18T13:42:46.746 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2024-08-18T13:42:46.747 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2024-08-18T13:42:46.748 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2024-08-18T13:42:46.748 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2024-08-18T13:42:47.948 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2024-08-18T13:42:47.948 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2024-08-18T13:42:47.949 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2024-08-18T13:42:47.950 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2024-08-18T13:42:47.951 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2024-08-18T13:42:47.952 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2024-08-18T13:42:47.953 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2024-08-18T13:42:47.954 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2024-08-18T13:42:49.182 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2024-08-18T13:42:49.183 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2024-08-18T13:42:49.184 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2024-08-18T13:42:49.185 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2024-08-18T13:42:49.186 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2024-08-18T13:42:49.187 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2024-08-18T13:42:49.188 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2024-08-18T13:42:49.366 INFO:tasks.ceph.mds.f.smithi178.stderr:2024-08-18T13:42:49.363+0000 7fc3d36e0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2024-08-18T13:42:50.387 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2024-08-18T13:42:50.388 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2024-08-18T13:42:50.389 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2024-08-18T13:42:50.390 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2024-08-18T13:42:50.391 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2024-08-18T13:42:50.391 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2024-08-18T13:42:50.391 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2024-08-18T13:42:50.391 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2024-08-18T13:42:50.391 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2024-08-18T13:42:50.391 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2024-08-18T13:42:50.392 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2024-08-18T13:42:50.393 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2024-08-18T13:42:50.394 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2024-08-18T13:42:50.394 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2024-08-18T13:42:50.394 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2024-08-18T13:42:50.394 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2024-08-18T13:42:50.394 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2024-08-18T13:42:50.394 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2024-08-18T13:42:51.581 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2024-08-18T13:42:51.581 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2024-08-18T13:42:51.581 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2024-08-18T13:42:51.581 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2024-08-18T13:42:51.581 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2024-08-18T13:42:51.581 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2024-08-18T13:42:51.582 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2024-08-18T13:42:51.583 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2024-08-18T13:42:51.584 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2024-08-18T13:42:51.585 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2024-08-18T13:42:51.586 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2024-08-18T13:42:51.587 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2024-08-18T13:42:51.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2024-08-18T13:42:51.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2024-08-18T13:42:51.588 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2024-08-18T13:42:52.839 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2024-08-18T13:42:52.840 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2024-08-18T13:42:52.841 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2024-08-18T13:42:52.842 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2024-08-18T13:42:52.843 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2024-08-18T13:42:52.844 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2024-08-18T13:42:52.845 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2024-08-18T13:42:53.891 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2024-08-18T13:42:53.892 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2024-08-18T13:42:53.893 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2024-08-18T13:42:53.894 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2024-08-18T13:42:53.895 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2024-08-18T13:42:53.896 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2024-08-18T13:42:53.897 INFO:tasks.workunit.client.0.smithi118.stdout:Take third snapshot .snap/test3 2024-08-18T13:42:53.937 INFO:tasks.workunit.client.0.smithi118.stdout:Delete the snapshots... 2024-08-18T13:42:56.591 INFO:tasks.workunit.client.0.smithi118.stdout:.//.snap/test1 2024-08-18T13:42:56.591 INFO:tasks.workunit.client.0.smithi118.stdout:.//.snap/test2 2024-08-18T13:42:56.591 INFO:tasks.workunit.client.0.smithi118.stdout:.//.snap/test3 2024-08-18T13:42:56.616 INFO:tasks.workunit.client.0.smithi118.stdout:./274/.snap/snap-subdir-test 2024-08-18T13:42:56.640 INFO:tasks.workunit.client.0.smithi118.stdout:./166/.snap/snap-subdir-test 2024-08-18T13:42:56.664 INFO:tasks.workunit.client.0.smithi118.stdout:./256/.snap/snap-subdir-test 2024-08-18T13:42:56.689 INFO:tasks.workunit.client.0.smithi118.stdout:./247/.snap/snap-subdir-test 2024-08-18T13:42:56.713 INFO:tasks.workunit.client.0.smithi118.stdout:./248/.snap/snap-subdir-test 2024-08-18T13:42:56.740 INFO:tasks.workunit.client.0.smithi118.stdout:./147/.snap/snap-subdir-test 2024-08-18T13:42:56.763 INFO:tasks.workunit.client.0.smithi118.stdout:./151/.snap/snap-subdir-test 2024-08-18T13:42:56.790 INFO:tasks.workunit.client.0.smithi118.stdout:./170/.snap/snap-subdir-test 2024-08-18T13:42:56.816 INFO:tasks.workunit.client.0.smithi118.stdout:./167/.snap/snap-subdir-test 2024-08-18T13:42:56.841 INFO:tasks.workunit.client.0.smithi118.stdout:./140/.snap/snap-subdir-test 2024-08-18T13:42:56.867 INFO:tasks.workunit.client.0.smithi118.stdout:./142/.snap/snap-subdir-test 2024-08-18T13:42:56.892 INFO:tasks.workunit.client.0.smithi118.stdout:./150/.snap/snap-subdir-test 2024-08-18T13:42:56.916 INFO:tasks.workunit.client.0.smithi118.stdout:./273/.snap/snap-subdir-test 2024-08-18T13:42:56.940 INFO:tasks.workunit.client.0.smithi118.stdout:./269/.snap/snap-subdir-test 2024-08-18T13:42:56.969 INFO:tasks.workunit.client.0.smithi118.stdout:./105/.snap/snap-subdir-test 2024-08-18T13:42:56.995 INFO:tasks.workunit.client.0.smithi118.stdout:./210/.snap/snap-subdir-test 2024-08-18T13:42:57.020 INFO:tasks.workunit.client.0.smithi118.stdout:./265/.snap/snap-subdir-test 2024-08-18T13:42:57.045 INFO:tasks.workunit.client.0.smithi118.stdout:./218/.snap/snap-subdir-test 2024-08-18T13:42:57.071 INFO:tasks.workunit.client.0.smithi118.stdout:./290/.snap/snap-subdir-test 2024-08-18T13:42:57.103 INFO:tasks.workunit.client.0.smithi118.stdout:./149/.snap/snap-subdir-test 2024-08-18T13:42:57.130 INFO:tasks.workunit.client.0.smithi118.stdout:./109/.snap/snap-subdir-test 2024-08-18T13:42:57.186 INFO:tasks.workunit.client.0.smithi118.stdout:./292/.snap/snap-subdir-test 2024-08-18T13:42:57.210 INFO:tasks.workunit.client.0.smithi118.stdout:./242/.snap/snap-subdir-test 2024-08-18T13:42:57.238 INFO:tasks.workunit.client.0.smithi118.stdout:./158/.snap/snap-subdir-test 2024-08-18T13:42:57.261 INFO:tasks.workunit.client.0.smithi118.stdout:./243/.snap/snap-subdir-test 2024-08-18T13:42:57.287 INFO:tasks.workunit.client.0.smithi118.stdout:./106/.snap/snap-subdir-test 2024-08-18T13:42:57.313 INFO:tasks.workunit.client.0.smithi118.stdout:./250/.snap/snap-subdir-test 2024-08-18T13:42:57.340 INFO:tasks.workunit.client.0.smithi118.stdout:./221/.snap/snap-subdir-test 2024-08-18T13:42:57.364 INFO:tasks.workunit.client.0.smithi118.stdout:./244/.snap/snap-subdir-test 2024-08-18T13:42:57.388 INFO:tasks.workunit.client.0.smithi118.stdout:./258/.snap/snap-subdir-test 2024-08-18T13:42:57.414 INFO:tasks.workunit.client.0.smithi118.stdout:./268/.snap/snap-subdir-test 2024-08-18T13:42:57.442 INFO:tasks.workunit.client.0.smithi118.stdout:./189/.snap/snap-subdir-test 2024-08-18T13:42:57.466 INFO:tasks.workunit.client.0.smithi118.stdout:./285/.snap/snap-subdir-test 2024-08-18T13:42:57.491 INFO:tasks.workunit.client.0.smithi118.stdout:./208/.snap/snap-subdir-test 2024-08-18T13:42:57.517 INFO:tasks.workunit.client.0.smithi118.stdout:./288/.snap/snap-subdir-test 2024-08-18T13:42:57.543 INFO:tasks.workunit.client.0.smithi118.stdout:./209/.snap/snap-subdir-test 2024-08-18T13:42:57.566 INFO:tasks.workunit.client.0.smithi118.stdout:./253/.snap/snap-subdir-test 2024-08-18T13:42:57.590 INFO:tasks.workunit.client.0.smithi118.stdout:./249/.snap/snap-subdir-test 2024-08-18T13:42:57.618 INFO:tasks.workunit.client.0.smithi118.stdout:./133/.snap/snap-subdir-test 2024-08-18T13:42:57.644 INFO:tasks.workunit.client.0.smithi118.stdout:./291/.snap/snap-subdir-test 2024-08-18T13:42:57.669 INFO:tasks.workunit.client.0.smithi118.stdout:./226/.snap/snap-subdir-test 2024-08-18T13:42:57.695 INFO:tasks.workunit.client.0.smithi118.stdout:./173/.snap/snap-subdir-test 2024-08-18T13:42:57.723 INFO:tasks.workunit.client.0.smithi118.stdout:./174/.snap/snap-subdir-test 2024-08-18T13:42:57.746 INFO:tasks.workunit.client.0.smithi118.stdout:./299/.snap/snap-subdir-test 2024-08-18T13:42:57.768 INFO:tasks.workunit.client.0.smithi118.stdout:./262/.snap/snap-subdir-test 2024-08-18T13:42:57.792 INFO:tasks.workunit.client.0.smithi118.stdout:./231/.snap/snap-subdir-test 2024-08-18T13:42:57.817 INFO:tasks.workunit.client.0.smithi118.stdout:./214/.snap/snap-subdir-test 2024-08-18T13:42:57.841 INFO:tasks.workunit.client.0.smithi118.stdout:./281/.snap/snap-subdir-test 2024-08-18T13:42:57.865 INFO:tasks.workunit.client.0.smithi118.stdout:./289/.snap/snap-subdir-test 2024-08-18T13:42:57.887 INFO:tasks.workunit.client.0.smithi118.stdout:./283/.snap/snap-subdir-test 2024-08-18T13:42:57.911 INFO:tasks.workunit.client.0.smithi118.stdout:./232/.snap/snap-subdir-test 2024-08-18T13:42:57.935 INFO:tasks.workunit.client.0.smithi118.stdout:./298/.snap/snap-subdir-test 2024-08-18T13:42:57.957 INFO:tasks.workunit.client.0.smithi118.stdout:./233/.snap/snap-subdir-test 2024-08-18T13:42:57.982 INFO:tasks.workunit.client.0.smithi118.stdout:./155/.snap/snap-subdir-test 2024-08-18T13:42:58.006 INFO:tasks.workunit.client.0.smithi118.stdout:./114/.snap/snap-subdir-test 2024-08-18T13:42:58.032 INFO:tasks.workunit.client.0.smithi118.stdout:./159/.snap/snap-subdir-test 2024-08-18T13:42:58.058 INFO:tasks.workunit.client.0.smithi118.stdout:./187/.snap/snap-subdir-test 2024-08-18T13:42:58.083 INFO:tasks.workunit.client.0.smithi118.stdout:./185/.snap/snap-subdir-test 2024-08-18T13:42:58.106 INFO:tasks.workunit.client.0.smithi118.stdout:./293/.snap/snap-subdir-test 2024-08-18T13:42:58.130 INFO:tasks.workunit.client.0.smithi118.stdout:./228/.snap/snap-subdir-test 2024-08-18T13:42:58.185 INFO:tasks.workunit.client.0.smithi118.stdout:./120/.snap/snap-subdir-test 2024-08-18T13:42:58.209 INFO:tasks.workunit.client.0.smithi118.stdout:./110/.snap/snap-subdir-test 2024-08-18T13:42:58.233 INFO:tasks.workunit.client.0.smithi118.stdout:./267/.snap/snap-subdir-test 2024-08-18T13:42:58.257 INFO:tasks.workunit.client.0.smithi118.stdout:./202/.snap/snap-subdir-test 2024-08-18T13:42:58.283 INFO:tasks.workunit.client.0.smithi118.stdout:./137/.snap/snap-subdir-test 2024-08-18T13:42:58.307 INFO:tasks.workunit.client.0.smithi118.stdout:./246/.snap/snap-subdir-test 2024-08-18T13:42:58.332 INFO:tasks.workunit.client.0.smithi118.stdout:./219/.snap/snap-subdir-test 2024-08-18T13:42:58.359 INFO:tasks.workunit.client.0.smithi118.stdout:./198/.snap/snap-subdir-test 2024-08-18T13:42:58.383 INFO:tasks.workunit.client.0.smithi118.stdout:./136/.snap/snap-subdir-test 2024-08-18T13:42:58.406 INFO:tasks.workunit.client.0.smithi118.stdout:./270/.snap/snap-subdir-test 2024-08-18T13:42:58.434 INFO:tasks.workunit.client.0.smithi118.stdout:./125/.snap/snap-subdir-test 2024-08-18T13:42:58.461 INFO:tasks.workunit.client.0.smithi118.stdout:./194/.snap/snap-subdir-test 2024-08-18T13:42:58.489 INFO:tasks.workunit.client.0.smithi118.stdout:./119/.snap/snap-subdir-test 2024-08-18T13:42:58.513 INFO:tasks.workunit.client.0.smithi118.stdout:./297/.snap/snap-subdir-test 2024-08-18T13:42:58.536 INFO:tasks.workunit.client.0.smithi118.stdout:./230/.snap/snap-subdir-test 2024-08-18T13:42:58.562 INFO:tasks.workunit.client.0.smithi118.stdout:./260/.snap/snap-subdir-test 2024-08-18T13:42:58.587 INFO:tasks.workunit.client.0.smithi118.stdout:./157/.snap/snap-subdir-test 2024-08-18T13:42:58.612 INFO:tasks.workunit.client.0.smithi118.stdout:./172/.snap/snap-subdir-test 2024-08-18T13:42:58.634 INFO:tasks.workunit.client.0.smithi118.stdout:./277/.snap/snap-subdir-test 2024-08-18T13:42:58.659 INFO:tasks.workunit.client.0.smithi118.stdout:./186/.snap/snap-subdir-test 2024-08-18T13:42:58.682 INFO:tasks.workunit.client.0.smithi118.stdout:./237/.snap/snap-subdir-test 2024-08-18T13:42:58.707 INFO:tasks.workunit.client.0.smithi118.stdout:./190/.snap/snap-subdir-test 2024-08-18T13:42:58.731 INFO:tasks.workunit.client.0.smithi118.stdout:./213/.snap/snap-subdir-test 2024-08-18T13:42:58.759 INFO:tasks.workunit.client.0.smithi118.stdout:./104/.snap/snap-subdir-test 2024-08-18T13:42:58.787 INFO:tasks.workunit.client.0.smithi118.stdout:./164/.snap/snap-subdir-test 2024-08-18T13:42:58.814 INFO:tasks.workunit.client.0.smithi118.stdout:./112/.snap/snap-subdir-test 2024-08-18T13:42:58.839 INFO:tasks.workunit.client.0.smithi118.stdout:./295/.snap/snap-subdir-test 2024-08-18T13:42:58.863 INFO:tasks.workunit.client.0.smithi118.stdout:./272/.snap/snap-subdir-test 2024-08-18T13:42:58.891 INFO:tasks.workunit.client.0.smithi118.stdout:./100/.snap/snap-subdir-test 2024-08-18T13:42:58.918 INFO:tasks.workunit.client.0.smithi118.stdout:./134/.snap/snap-subdir-test 2024-08-18T13:42:58.943 INFO:tasks.workunit.client.0.smithi118.stdout:./200/.snap/snap-subdir-test 2024-08-18T13:42:58.971 INFO:tasks.workunit.client.0.smithi118.stdout:./115/.snap/snap-subdir-test 2024-08-18T13:42:58.994 INFO:tasks.workunit.client.0.smithi118.stdout:./284/.snap/snap-subdir-test 2024-08-18T13:42:59.018 INFO:tasks.workunit.client.0.smithi118.stdout:./280/.snap/snap-subdir-test 2024-08-18T13:42:59.046 INFO:tasks.workunit.client.0.smithi118.stdout:./116/.snap/snap-subdir-test 2024-08-18T13:42:59.071 INFO:tasks.workunit.client.0.smithi118.stdout:./245/.snap/snap-subdir-test 2024-08-18T13:42:59.099 INFO:tasks.workunit.client.0.smithi118.stdout:./195/.snap/snap-subdir-test 2024-08-18T13:42:59.126 INFO:tasks.workunit.client.0.smithi118.stdout:./138/.snap/snap-subdir-test 2024-08-18T13:42:59.154 INFO:tasks.workunit.client.0.smithi118.stdout:./135/.snap/snap-subdir-test 2024-08-18T13:42:59.201 INFO:tasks.workunit.client.0.smithi118.stdout:./117/.snap/snap-subdir-test 2024-08-18T13:42:59.223 INFO:tasks.workunit.client.0.smithi118.stdout:./241/.snap/snap-subdir-test 2024-08-18T13:42:59.251 INFO:tasks.workunit.client.0.smithi118.stdout:./143/.snap/snap-subdir-test 2024-08-18T13:42:59.277 INFO:tasks.workunit.client.0.smithi118.stdout:./180/.snap/snap-subdir-test 2024-08-18T13:42:59.304 INFO:tasks.workunit.client.0.smithi118.stdout:./126/.snap/snap-subdir-test 2024-08-18T13:42:59.331 INFO:tasks.workunit.client.0.smithi118.stdout:./161/.snap/snap-subdir-test 2024-08-18T13:42:59.357 INFO:tasks.workunit.client.0.smithi118.stdout:./204/.snap/snap-subdir-test 2024-08-18T13:42:59.385 INFO:tasks.workunit.client.0.smithi118.stdout:./176/.snap/snap-subdir-test 2024-08-18T13:42:59.410 INFO:tasks.workunit.client.0.smithi118.stdout:./255/.snap/snap-subdir-test 2024-08-18T13:42:59.433 INFO:tasks.workunit.client.0.smithi118.stdout:./215/.snap/snap-subdir-test 2024-08-18T13:42:59.460 INFO:tasks.workunit.client.0.smithi118.stdout:./111/.snap/snap-subdir-test 2024-08-18T13:42:59.474 INFO:tasks.workunit.client.0.smithi118.stdout:./238/.snap/snap-subdir-test 2024-08-18T13:42:59.497 INFO:tasks.workunit.client.0.smithi118.stdout:./191/.snap/snap-subdir-test 2024-08-18T13:42:59.522 INFO:tasks.workunit.client.0.smithi118.stdout:./184/.snap/snap-subdir-test 2024-08-18T13:42:59.549 INFO:tasks.workunit.client.0.smithi118.stdout:./192/.snap/snap-subdir-test 2024-08-18T13:42:59.578 INFO:tasks.workunit.client.0.smithi118.stdout:./103/.snap/snap-subdir-test 2024-08-18T13:42:59.606 INFO:tasks.workunit.client.0.smithi118.stdout:./128/.snap/snap-subdir-test 2024-08-18T13:42:59.630 INFO:tasks.workunit.client.0.smithi118.stdout:./229/.snap/snap-subdir-test 2024-08-18T13:42:59.658 INFO:tasks.workunit.client.0.smithi118.stdout:./144/.snap/snap-subdir-test 2024-08-18T13:42:59.685 INFO:tasks.workunit.client.0.smithi118.stdout:./146/.snap/snap-subdir-test 2024-08-18T13:42:59.710 INFO:tasks.workunit.client.0.smithi118.stdout:./227/.snap/snap-subdir-test 2024-08-18T13:42:59.736 INFO:tasks.workunit.client.0.smithi118.stdout:./278/.snap/snap-subdir-test 2024-08-18T13:42:59.759 INFO:tasks.workunit.client.0.smithi118.stdout:./141/.snap/snap-subdir-test 2024-08-18T13:42:59.777 INFO:tasks.workunit.client.0.smithi118.stdout:./234/.snap/snap-subdir-test 2024-08-18T13:42:59.793 INFO:tasks.workunit.client.0.smithi118.stdout:./225/.snap/snap-subdir-test 2024-08-18T13:42:59.811 INFO:tasks.workunit.client.0.smithi118.stdout:./182/.snap/snap-subdir-test 2024-08-18T13:42:59.829 INFO:tasks.workunit.client.0.smithi118.stdout:./107/.snap/snap-subdir-test 2024-08-18T13:42:59.851 INFO:tasks.workunit.client.0.smithi118.stdout:./178/.snap/snap-subdir-test 2024-08-18T13:42:59.874 INFO:tasks.workunit.client.0.smithi118.stdout:./266/.snap/snap-subdir-test 2024-08-18T13:42:59.899 INFO:tasks.workunit.client.0.smithi118.stdout:./240/.snap/snap-subdir-test 2024-08-18T13:42:59.926 INFO:tasks.workunit.client.0.smithi118.stdout:./156/.snap/snap-subdir-test 2024-08-18T13:42:59.953 INFO:tasks.workunit.client.0.smithi118.stdout:./131/.snap/snap-subdir-test 2024-08-18T13:42:59.979 INFO:tasks.workunit.client.0.smithi118.stdout:./123/.snap/snap-subdir-test 2024-08-18T13:43:00.005 INFO:tasks.workunit.client.0.smithi118.stdout:./160/.snap/snap-subdir-test 2024-08-18T13:43:00.030 INFO:tasks.workunit.client.0.smithi118.stdout:./165/.snap/snap-subdir-test 2024-08-18T13:43:00.055 INFO:tasks.workunit.client.0.smithi118.stdout:./101/.snap/snap-subdir-test 2024-08-18T13:43:00.081 INFO:tasks.workunit.client.0.smithi118.stdout:./148/.snap/snap-subdir-test 2024-08-18T13:43:00.106 INFO:tasks.workunit.client.0.smithi118.stdout:./168/.snap/snap-subdir-test 2024-08-18T13:43:00.132 INFO:tasks.workunit.client.0.smithi118.stdout:./179/.snap/snap-subdir-test 2024-08-18T13:43:00.155 INFO:tasks.workunit.client.0.smithi118.stdout:./257/.snap/snap-subdir-test 2024-08-18T13:43:00.186 INFO:tasks.workunit.client.0.smithi118.stdout:./252/.snap/snap-subdir-test 2024-08-18T13:43:00.211 INFO:tasks.workunit.client.0.smithi118.stdout:./132/.snap/snap-subdir-test 2024-08-18T13:43:00.234 INFO:tasks.workunit.client.0.smithi118.stdout:./236/.snap/snap-subdir-test 2024-08-18T13:43:00.259 INFO:tasks.workunit.client.0.smithi118.stdout:./211/.snap/snap-subdir-test 2024-08-18T13:43:00.286 INFO:tasks.workunit.client.0.smithi118.stdout:./199/.snap/snap-subdir-test 2024-08-18T13:43:00.310 INFO:tasks.workunit.client.0.smithi118.stdout:./205/.snap/snap-subdir-test 2024-08-18T13:43:00.335 INFO:tasks.workunit.client.0.smithi118.stdout:./163/.snap/snap-subdir-test 2024-08-18T13:43:00.358 INFO:tasks.workunit.client.0.smithi118.stdout:./261/.snap/snap-subdir-test 2024-08-18T13:43:00.381 INFO:tasks.workunit.client.0.smithi118.stdout:./201/.snap/snap-subdir-test 2024-08-18T13:43:00.404 INFO:tasks.workunit.client.0.smithi118.stdout:./282/.snap/snap-subdir-test 2024-08-18T13:43:00.429 INFO:tasks.workunit.client.0.smithi118.stdout:./296/.snap/snap-subdir-test 2024-08-18T13:43:00.453 INFO:tasks.workunit.client.0.smithi118.stdout:./217/.snap/snap-subdir-test 2024-08-18T13:43:00.482 INFO:tasks.workunit.client.0.smithi118.stdout:./153/.snap/snap-subdir-test 2024-08-18T13:43:00.508 INFO:tasks.workunit.client.0.smithi118.stdout:./206/.snap/snap-subdir-test 2024-08-18T13:43:00.535 INFO:tasks.workunit.client.0.smithi118.stdout:./122/.snap/snap-subdir-test 2024-08-18T13:43:00.560 INFO:tasks.workunit.client.0.smithi118.stdout:./162/.snap/snap-subdir-test 2024-08-18T13:43:00.587 INFO:tasks.workunit.client.0.smithi118.stdout:./197/.snap/snap-subdir-test 2024-08-18T13:43:00.611 INFO:tasks.workunit.client.0.smithi118.stdout:./216/.snap/snap-subdir-test 2024-08-18T13:43:02.670 INFO:tasks.workunit.client.0.smithi118.stdout:./254/.snap/snap-subdir-test 2024-08-18T13:43:02.697 INFO:tasks.workunit.client.0.smithi118.stdout:./113/.snap/snap-subdir-test 2024-08-18T13:43:02.725 INFO:tasks.workunit.client.0.smithi118.stdout:./152/.snap/snap-subdir-test 2024-08-18T13:43:02.754 INFO:tasks.workunit.client.0.smithi118.stdout:./121/.snap/snap-subdir-test 2024-08-18T13:43:02.778 INFO:tasks.workunit.client.0.smithi118.stdout:./118/.snap/snap-subdir-test 2024-08-18T13:43:02.806 INFO:tasks.workunit.client.0.smithi118.stdout:./196/.snap/snap-subdir-test 2024-08-18T13:43:02.837 INFO:tasks.workunit.client.0.smithi118.stdout:./124/.snap/snap-subdir-test 2024-08-18T13:43:02.864 INFO:tasks.workunit.client.0.smithi118.stdout:./239/.snap/snap-subdir-test 2024-08-18T13:43:02.890 INFO:tasks.workunit.client.0.smithi118.stdout:./188/.snap/snap-subdir-test 2024-08-18T13:43:02.915 INFO:tasks.workunit.client.0.smithi118.stdout:./287/.snap/snap-subdir-test 2024-08-18T13:43:02.941 INFO:tasks.workunit.client.0.smithi118.stdout:./275/.snap/snap-subdir-test 2024-08-18T13:43:02.966 INFO:tasks.workunit.client.0.smithi118.stdout:./259/.snap/snap-subdir-test 2024-08-18T13:43:02.992 INFO:tasks.workunit.client.0.smithi118.stdout:./224/.snap/snap-subdir-test 2024-08-18T13:43:03.017 INFO:tasks.workunit.client.0.smithi118.stdout:./264/.snap/snap-subdir-test 2024-08-18T13:43:03.045 INFO:tasks.workunit.client.0.smithi118.stdout:./145/.snap/snap-subdir-test 2024-08-18T13:43:03.071 INFO:tasks.workunit.client.0.smithi118.stdout:./279/.snap/snap-subdir-test 2024-08-18T13:43:03.095 INFO:tasks.workunit.client.0.smithi118.stdout:./263/.snap/snap-subdir-test 2024-08-18T13:43:03.124 INFO:tasks.workunit.client.0.smithi118.stdout:./130/.snap/snap-subdir-test 2024-08-18T13:43:03.153 INFO:tasks.workunit.client.0.smithi118.stdout:./193/.snap/snap-subdir-test 2024-08-18T13:43:03.180 INFO:tasks.workunit.client.0.smithi118.stdout:./223/.snap/snap-subdir-test 2024-08-18T13:43:03.205 INFO:tasks.workunit.client.0.smithi118.stdout:./207/.snap/snap-subdir-test 2024-08-18T13:43:03.231 INFO:tasks.workunit.client.0.smithi118.stdout:./276/.snap/snap-subdir-test 2024-08-18T13:43:03.257 INFO:tasks.workunit.client.0.smithi118.stdout:./271/.snap/snap-subdir-test 2024-08-18T13:43:03.282 INFO:tasks.workunit.client.0.smithi118.stdout:./251/.snap/snap-subdir-test 2024-08-18T13:43:03.310 INFO:tasks.workunit.client.0.smithi118.stdout:./175/.snap/snap-subdir-test 2024-08-18T13:43:03.334 INFO:tasks.workunit.client.0.smithi118.stdout:./222/.snap/snap-subdir-test 2024-08-18T13:43:03.358 INFO:tasks.workunit.client.0.smithi118.stdout:./220/.snap/snap-subdir-test 2024-08-18T13:43:03.381 INFO:tasks.workunit.client.0.smithi118.stdout:./203/.snap/snap-subdir-test 2024-08-18T13:43:03.406 INFO:tasks.workunit.client.0.smithi118.stdout:./181/.snap/snap-subdir-test 2024-08-18T13:43:03.427 INFO:tasks.workunit.client.0.smithi118.stdout:./286/.snap/snap-subdir-test 2024-08-18T13:43:03.448 INFO:tasks.workunit.client.0.smithi118.stdout:./127/.snap/snap-subdir-test 2024-08-18T13:43:03.476 INFO:tasks.workunit.client.0.smithi118.stdout:./108/.snap/snap-subdir-test 2024-08-18T13:43:03.506 INFO:tasks.workunit.client.0.smithi118.stdout:./183/.snap/snap-subdir-test 2024-08-18T13:43:03.529 INFO:tasks.workunit.client.0.smithi118.stdout:./294/.snap/snap-subdir-test 2024-08-18T13:43:03.552 INFO:tasks.workunit.client.0.smithi118.stdout:./235/.snap/snap-subdir-test 2024-08-18T13:43:03.578 INFO:tasks.workunit.client.0.smithi118.stdout:./177/.snap/snap-subdir-test 2024-08-18T13:43:03.605 INFO:tasks.workunit.client.0.smithi118.stdout:./169/.snap/snap-subdir-test 2024-08-18T13:43:03.630 INFO:tasks.workunit.client.0.smithi118.stdout:./212/.snap/snap-subdir-test 2024-08-18T13:43:03.661 INFO:tasks.workunit.client.0.smithi118.stdout:./154/.snap/snap-subdir-test 2024-08-18T13:43:03.690 INFO:tasks.workunit.client.0.smithi118.stdout:./171/.snap/snap-subdir-test 2024-08-18T13:43:03.720 INFO:tasks.workunit.client.0.smithi118.stdout:./139/.snap/snap-subdir-test 2024-08-18T13:43:03.749 INFO:tasks.workunit.client.0.smithi118.stdout:./102/.snap/snap-subdir-test 2024-08-18T13:43:03.777 INFO:tasks.workunit.client.0.smithi118.stdout:./129/.snap/snap-subdir-test 2024-08-18T13:43:11.699 INFO:tasks.workunit.client.0.smithi118.stdout:Delete all the files and directories ... 2024-08-18T13:43:12.374 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:43:12.376 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:43:12.994 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/19' 2024-08-18T13:43:12.994 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/15' 2024-08-18T13:43:12.994 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/31' 2024-08-18T13:43:12.994 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/36' 2024-08-18T13:43:12.994 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/20' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/12' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/39' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/10' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/34' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/38' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/11' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/16' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/32' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/30' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/14' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/18' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/13' 2024-08-18T13:43:12.995 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/33' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/17' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/35' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './100/37' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './100' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/19' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/15' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/31' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/36' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/20' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/12' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/39' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/10' 2024-08-18T13:43:12.996 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/34' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/38' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/11' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/16' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/32' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/30' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/14' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/18' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/13' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/33' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/17' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/35' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed './101/37' 2024-08-18T13:43:12.997 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './101' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/19' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/15' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/31' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/36' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/20' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/12' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/39' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/10' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/34' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/38' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/11' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/16' 2024-08-18T13:43:12.998 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/32' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/30' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/14' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/18' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/13' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/33' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/17' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/35' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './102/37' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './102' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/19' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/15' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/31' 2024-08-18T13:43:12.999 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/36' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/20' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/12' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/39' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/10' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/34' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/38' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/11' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/16' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/32' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/30' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/14' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/18' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/13' 2024-08-18T13:43:13.000 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/33' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/17' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/35' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './103/37' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './103' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/19' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/15' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/31' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/36' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/20' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/12' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/39' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/10' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/34' 2024-08-18T13:43:13.001 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/38' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/11' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/16' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/32' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/30' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/14' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/18' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/13' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/33' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/17' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/35' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './104/37' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './104' 2024-08-18T13:43:13.002 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/19' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/15' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/31' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/36' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/20' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/12' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/39' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/10' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/34' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/38' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/11' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/16' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/32' 2024-08-18T13:43:13.003 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/30' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/14' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/18' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/13' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/33' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/17' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/35' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './105/37' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './105' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/19' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/15' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/31' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/36' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/20' 2024-08-18T13:43:13.004 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/12' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/39' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/10' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/34' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/38' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/11' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/16' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/32' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/30' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/14' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/18' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/13' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/33' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/17' 2024-08-18T13:43:13.005 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/35' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './106/37' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './106' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/19' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/15' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/31' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/36' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/20' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/12' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/39' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/10' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/34' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/38' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/11' 2024-08-18T13:43:13.006 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/16' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/32' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/30' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/14' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/18' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/13' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/33' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/17' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/35' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './107/37' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './107' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/19' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/15' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/31' 2024-08-18T13:43:13.007 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/36' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/20' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/12' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/39' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/10' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/34' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/38' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/11' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/16' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/32' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/30' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/14' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/18' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/13' 2024-08-18T13:43:13.008 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/33' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/17' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/35' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './108/37' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './108' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/19' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/15' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/31' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/36' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/20' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/12' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/39' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/10' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/34' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/38' 2024-08-18T13:43:13.009 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/11' 2024-08-18T13:43:13.010 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/16' 2024-08-18T13:43:13.010 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/32' 2024-08-18T13:43:13.010 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/30' 2024-08-18T13:43:13.932 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/14' 2024-08-18T13:43:13.932 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/18' 2024-08-18T13:43:13.932 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/13' 2024-08-18T13:43:13.932 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/33' 2024-08-18T13:43:13.932 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/17' 2024-08-18T13:43:13.932 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/35' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './109/37' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './109' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/19' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/15' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/31' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/36' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/20' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/12' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/39' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/10' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/34' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/38' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/11' 2024-08-18T13:43:13.933 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/16' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/32' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/30' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/14' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/18' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/13' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/33' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/17' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/35' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './110/37' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './110' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/19' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/15' 2024-08-18T13:43:13.934 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/31' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/36' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/20' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/12' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/39' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/10' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/34' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/38' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/11' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/16' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/32' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/30' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/14' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/18' 2024-08-18T13:43:13.935 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/13' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/33' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/17' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/35' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './111/37' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './111' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/19' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/15' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/31' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/36' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/20' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/12' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/39' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/10' 2024-08-18T13:43:13.936 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/34' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/38' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/11' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/16' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/32' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/30' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/14' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/18' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/13' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/33' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/17' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/35' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './112/37' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './112' 2024-08-18T13:43:13.937 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/19' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/15' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/31' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/36' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/20' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/12' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/39' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/10' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/34' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/38' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/11' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/16' 2024-08-18T13:43:13.938 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/32' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/30' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/14' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/18' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/13' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/33' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/17' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/35' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './113/37' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './113' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/19' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/15' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/31' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/36' 2024-08-18T13:43:13.939 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/20' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/12' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/39' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/10' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/34' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/38' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/11' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/16' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/32' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/30' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/14' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/18' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/13' 2024-08-18T13:43:13.940 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/33' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/17' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/35' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './114/37' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './114' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/19' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/15' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/31' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/36' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/20' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/12' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/39' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/10' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/34' 2024-08-18T13:43:13.941 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/38' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/11' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/16' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/32' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/30' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/14' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/18' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/13' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/33' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/17' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/35' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './115/37' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './115' 2024-08-18T13:43:13.942 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/19' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/15' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/31' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/36' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/20' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/12' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/39' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/10' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/34' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/38' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/11' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/16' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/32' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/30' 2024-08-18T13:43:13.943 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/14' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/18' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/13' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/33' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/17' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/35' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './116/37' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './116' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/19' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/15' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/31' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/36' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/20' 2024-08-18T13:43:13.944 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/12' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/39' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/10' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/34' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/38' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/11' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/16' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/32' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/30' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/14' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/18' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/13' 2024-08-18T13:43:13.945 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/33' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/17' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/35' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './117/37' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './117' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/19' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/15' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/31' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/36' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/20' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/12' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/39' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/10' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/34' 2024-08-18T13:43:13.946 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/38' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/11' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/16' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/32' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/30' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/14' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/18' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/13' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/33' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/17' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/35' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './118/37' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './118' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/19' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/15' 2024-08-18T13:43:13.947 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/31' 2024-08-18T13:43:13.948 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/36' 2024-08-18T13:43:13.948 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/20' 2024-08-18T13:43:13.948 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/12' 2024-08-18T13:43:14.971 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/39' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/10' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/34' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/38' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/11' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/16' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/32' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/30' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/14' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/18' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/13' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/33' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/17' 2024-08-18T13:43:14.972 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/35' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './119/37' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './119' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/19' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/15' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/31' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/36' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/20' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/12' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/39' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/10' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/34' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/38' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/11' 2024-08-18T13:43:14.973 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/16' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/32' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/30' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/14' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/18' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/13' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/33' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/17' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/35' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './120/37' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './120' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/19' 2024-08-18T13:43:14.974 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/15' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/31' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/36' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/20' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/12' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/39' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/10' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/34' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/38' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/11' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/16' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/32' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/30' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/14' 2024-08-18T13:43:14.975 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/18' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/13' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/33' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/17' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/35' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './121/37' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './121' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/19' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/15' 2024-08-18T13:43:14.976 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/31' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/36' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/20' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/12' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/39' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/10' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/34' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/38' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/11' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/16' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/32' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/30' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/14' 2024-08-18T13:43:14.977 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/18' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/13' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/33' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/17' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/35' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './122/37' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './122' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/19' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/15' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/31' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/36' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/20' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/12' 2024-08-18T13:43:14.978 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/39' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/10' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/34' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/38' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/11' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/16' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/32' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/30' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/14' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/18' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/13' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/33' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/17' 2024-08-18T13:43:14.979 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/35' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './123/37' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './123' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/19' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/15' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/31' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/36' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/20' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/12' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/39' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/10' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/34' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/38' 2024-08-18T13:43:14.980 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/11' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/16' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/32' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/30' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/14' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/18' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/13' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/33' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/17' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/35' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './124/37' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './124' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/19' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/15' 2024-08-18T13:43:14.981 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/31' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/36' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/20' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/12' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/39' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/10' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/34' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/38' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/11' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/16' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/32' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/30' 2024-08-18T13:43:14.982 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/14' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/18' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/13' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/33' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/17' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/35' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './125/37' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './125' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/19' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/15' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/31' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/36' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/20' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/12' 2024-08-18T13:43:14.983 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/39' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/10' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/34' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/38' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/11' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/16' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/32' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/30' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/14' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/18' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/13' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/33' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/17' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/35' 2024-08-18T13:43:14.984 INFO:tasks.workunit.client.0.smithi118.stdout:removed './126/37' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './126' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/19' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/15' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/31' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/36' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/20' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/12' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/39' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/10' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/34' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/38' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/11' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/16' 2024-08-18T13:43:14.985 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/32' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/30' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/14' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/18' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/13' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/33' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/17' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/35' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './127/37' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './127' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/19' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/15' 2024-08-18T13:43:14.986 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/31' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/36' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/20' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/12' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/39' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/10' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/34' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/38' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/11' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/16' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/32' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/30' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/14' 2024-08-18T13:43:14.987 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/18' 2024-08-18T13:43:14.988 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/13' 2024-08-18T13:43:14.988 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/33' 2024-08-18T13:43:14.988 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/17' 2024-08-18T13:43:14.988 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/35' 2024-08-18T13:43:16.052 INFO:tasks.workunit.client.0.smithi118.stdout:removed './128/37' 2024-08-18T13:43:16.052 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './128' 2024-08-18T13:43:16.052 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/19' 2024-08-18T13:43:16.052 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/15' 2024-08-18T13:43:16.052 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/31' 2024-08-18T13:43:16.052 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/36' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/20' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/12' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/39' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/10' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/34' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/38' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/11' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/16' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/32' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/30' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/14' 2024-08-18T13:43:16.053 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/18' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/13' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/33' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/17' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/35' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './129/37' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './129' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/19' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/15' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/31' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/36' 2024-08-18T13:43:16.054 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/20' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/12' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/39' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/10' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/34' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/38' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/11' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/16' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/32' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/30' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/14' 2024-08-18T13:43:16.055 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/18' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/13' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/33' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/17' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/35' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './130/37' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './130' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/19' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/15' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/31' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/36' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/20' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/12' 2024-08-18T13:43:16.056 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/39' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/10' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/34' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/38' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/11' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/16' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/32' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/30' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/14' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/18' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/13' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/33' 2024-08-18T13:43:16.057 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/17' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/35' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './131/37' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './131' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/19' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/15' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/31' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/36' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/20' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/12' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/39' 2024-08-18T13:43:16.058 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/10' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/34' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/38' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/11' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/16' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/32' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/30' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/14' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/18' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/13' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/33' 2024-08-18T13:43:16.059 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/17' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/35' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './132/37' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './132' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/19' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/15' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/31' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/36' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/20' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/12' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/39' 2024-08-18T13:43:16.060 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/10' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/34' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/38' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/11' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/16' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/32' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/30' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/14' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/18' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/13' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/33' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/17' 2024-08-18T13:43:16.061 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/35' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './133/37' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './133' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/19' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/15' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/31' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/36' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/20' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/12' 2024-08-18T13:43:16.062 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/39' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/10' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/34' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/38' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/11' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/16' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/32' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/30' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/14' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/18' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/13' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/33' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/17' 2024-08-18T13:43:16.063 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/35' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './134/37' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './134' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/19' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/15' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/31' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/36' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/20' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/12' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/39' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/10' 2024-08-18T13:43:16.064 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/34' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/38' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/11' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/16' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/32' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/30' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/14' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/18' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/13' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/33' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/17' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/35' 2024-08-18T13:43:16.065 INFO:tasks.workunit.client.0.smithi118.stdout:removed './135/37' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './135' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/19' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/15' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/31' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/36' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/20' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/12' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/39' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/10' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/34' 2024-08-18T13:43:16.066 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/38' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/11' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/16' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/32' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/30' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/14' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/18' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/13' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/33' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/17' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/35' 2024-08-18T13:43:16.067 INFO:tasks.workunit.client.0.smithi118.stdout:removed './136/37' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './136' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/19' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/15' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/31' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/36' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/20' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/12' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/39' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/10' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/34' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/38' 2024-08-18T13:43:16.068 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/11' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/16' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/32' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/30' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/14' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/18' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/13' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/33' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/17' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/35' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed './137/37' 2024-08-18T13:43:16.069 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './137' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/19' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/15' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/31' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/36' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/20' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/12' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/39' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/10' 2024-08-18T13:43:16.070 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/34' 2024-08-18T13:43:16.071 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/38' 2024-08-18T13:43:16.071 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/11' 2024-08-18T13:43:16.071 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/16' 2024-08-18T13:43:17.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/32' 2024-08-18T13:43:17.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/30' 2024-08-18T13:43:17.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/14' 2024-08-18T13:43:17.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/18' 2024-08-18T13:43:17.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/13' 2024-08-18T13:43:17.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/33' 2024-08-18T13:43:17.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/17' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/35' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './138/37' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './138' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/19' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/15' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/31' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/36' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/20' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/12' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/39' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/10' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/34' 2024-08-18T13:43:17.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/38' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/11' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/16' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/32' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/30' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/14' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/18' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/13' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/33' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/17' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/35' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './139/37' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './139' 2024-08-18T13:43:17.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/19' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/15' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/31' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/36' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/20' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/12' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/39' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/10' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/34' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/38' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/11' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/16' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/32' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/30' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/14' 2024-08-18T13:43:17.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/18' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/13' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/33' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/17' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/35' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './140/37' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './140' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/19' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/15' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/31' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/36' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/20' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/12' 2024-08-18T13:43:17.128 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/39' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/10' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/34' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/38' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/11' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/16' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/32' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/30' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/14' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/18' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/13' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/33' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/17' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/35' 2024-08-18T13:43:17.129 INFO:tasks.workunit.client.0.smithi118.stdout:removed './141/37' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './141' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/19' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/15' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/31' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/36' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/20' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/12' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/39' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/10' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/34' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/38' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/11' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/16' 2024-08-18T13:43:17.130 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/32' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/30' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/14' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/18' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/13' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/33' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/17' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/35' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './142/37' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './142' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/19' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/15' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/31' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/36' 2024-08-18T13:43:17.131 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/20' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/12' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/39' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/10' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/34' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/38' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/11' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/16' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/32' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/30' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/14' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/18' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/13' 2024-08-18T13:43:17.132 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/33' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/17' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/35' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './143/37' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './143' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/19' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/15' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/31' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/36' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/20' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/12' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/39' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/10' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/34' 2024-08-18T13:43:17.133 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/38' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/11' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/16' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/32' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/30' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/14' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/18' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/13' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/33' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/17' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/35' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './144/37' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './144' 2024-08-18T13:43:17.134 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/19' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/15' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/31' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/36' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/20' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/12' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/39' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/10' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/34' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/38' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/11' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/16' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/32' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/30' 2024-08-18T13:43:17.135 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/14' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/18' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/13' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/33' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/17' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/35' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './145/37' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './145' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/19' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/15' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/31' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/36' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/20' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/12' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/39' 2024-08-18T13:43:17.136 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/10' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/34' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/38' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/11' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/16' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/32' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/30' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/14' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/18' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/13' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/33' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/17' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/35' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed './146/37' 2024-08-18T13:43:17.137 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './146' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/19' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/15' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/31' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/36' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/20' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/12' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/39' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/10' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/34' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/38' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/11' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/16' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/32' 2024-08-18T13:43:17.138 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/30' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/14' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/18' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/13' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/33' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/17' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/35' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './147/37' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './147' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/19' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/15' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/31' 2024-08-18T13:43:17.139 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/36' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/20' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/12' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/39' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/10' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/34' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/38' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/11' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/16' 2024-08-18T13:43:18.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/32' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/30' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/14' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/18' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/13' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/33' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/17' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/35' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './148/37' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './148' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/19' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/15' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/31' 2024-08-18T13:43:18.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/36' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/20' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/12' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/39' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/10' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/34' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/38' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/11' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/16' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/32' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/30' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/14' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/18' 2024-08-18T13:43:18.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/13' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/33' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/17' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/35' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './149/37' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './149' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/19' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/15' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/31' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/36' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/20' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/12' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/39' 2024-08-18T13:43:18.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/10' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/34' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/38' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/11' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/16' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/32' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/30' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/14' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/18' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/13' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/33' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/17' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/35' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './150/37' 2024-08-18T13:43:18.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './150' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/19' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/15' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/31' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/36' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/20' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/12' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/39' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/10' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/34' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/38' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/11' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/16' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/32' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/30' 2024-08-18T13:43:18.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/14' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/18' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/13' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/33' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/17' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/35' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './151/37' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './151' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/19' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/15' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/31' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/36' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/20' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/12' 2024-08-18T13:43:18.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/39' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/10' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/34' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/38' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/11' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/16' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/32' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/30' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/14' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/18' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/13' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/33' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/17' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/35' 2024-08-18T13:43:18.195 INFO:tasks.workunit.client.0.smithi118.stdout:removed './152/37' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './152' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/19' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/15' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/31' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/36' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/20' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/12' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/39' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/10' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/34' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/38' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/11' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/16' 2024-08-18T13:43:18.196 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/32' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/30' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/14' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/18' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/13' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/33' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/17' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/35' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './153/37' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './153' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/19' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/15' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/31' 2024-08-18T13:43:18.197 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/36' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/20' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/12' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/39' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/10' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/34' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/38' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/11' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/16' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/32' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/30' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/14' 2024-08-18T13:43:18.198 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/18' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/13' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/33' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/17' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/35' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './154/37' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './154' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/19' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/15' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/31' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/36' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/20' 2024-08-18T13:43:18.199 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/12' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/39' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/10' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/34' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/38' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/11' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/16' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/32' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/30' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/14' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/18' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/13' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/33' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/17' 2024-08-18T13:43:18.200 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/35' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './155/37' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './155' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/19' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/15' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/31' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/36' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/20' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/12' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/39' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/10' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/34' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/38' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/11' 2024-08-18T13:43:18.201 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/16' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/32' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/30' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/14' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/18' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/13' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/33' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/17' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/35' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './156/37' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './156' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/19' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/15' 2024-08-18T13:43:18.202 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/31' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/36' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/20' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/12' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/39' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/10' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/34' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/38' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/11' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/16' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/32' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/30' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/14' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/18' 2024-08-18T13:43:18.203 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/13' 2024-08-18T13:43:18.204 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/33' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/17' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/35' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './157/37' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './157' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/19' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/15' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/31' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/36' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/20' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/12' 2024-08-18T13:43:19.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/39' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/10' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/34' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/38' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/11' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/16' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/32' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/30' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/14' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/18' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/13' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/33' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/17' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/35' 2024-08-18T13:43:19.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './158/37' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './158' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/19' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/15' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/31' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/36' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/20' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/12' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/39' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/10' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/34' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/38' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/11' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/16' 2024-08-18T13:43:19.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/32' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/30' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/14' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/18' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/13' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/33' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/17' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/35' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './159/37' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './159' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/19' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/15' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/31' 2024-08-18T13:43:19.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/36' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/20' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/12' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/39' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/10' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/34' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/38' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/11' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/16' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/32' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/30' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/14' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/18' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/13' 2024-08-18T13:43:19.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/33' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/17' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/35' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './160/37' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './160' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/19' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/15' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/31' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/36' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/20' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/12' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/39' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/10' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/34' 2024-08-18T13:43:19.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/38' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/11' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/16' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/32' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/30' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/14' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/18' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/13' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/33' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/17' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/35' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './161/37' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './161' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/19' 2024-08-18T13:43:19.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/15' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/31' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/36' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/20' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/12' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/39' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/10' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/34' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/38' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/11' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/16' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/32' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/30' 2024-08-18T13:43:19.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/14' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/18' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/13' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/33' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/17' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/35' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './162/37' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './162' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/19' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/15' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/31' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/36' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/20' 2024-08-18T13:43:19.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/12' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/39' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/10' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/34' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/38' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/11' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/16' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/32' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/30' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/14' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/18' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/13' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/33' 2024-08-18T13:43:19.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/17' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/35' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './163/37' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './163' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/19' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/15' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/31' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/36' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/20' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/12' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/39' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/10' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/34' 2024-08-18T13:43:19.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/38' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/11' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/16' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/32' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/30' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/14' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/18' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/13' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/33' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/17' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/35' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './164/37' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './164' 2024-08-18T13:43:19.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/19' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/15' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/31' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/36' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/20' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/12' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/39' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/10' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/34' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/38' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/11' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/16' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/32' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/30' 2024-08-18T13:43:19.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/14' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/18' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/13' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/33' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/17' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/35' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './165/37' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './165' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/19' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/15' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/31' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/36' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/20' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/12' 2024-08-18T13:43:19.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/39' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/10' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/34' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/38' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/11' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/16' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/32' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/30' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/14' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/18' 2024-08-18T13:43:19.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/13' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/33' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/17' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/35' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './166/37' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './166' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/19' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/15' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/31' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/36' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/20' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/12' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/39' 2024-08-18T13:43:19.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/10' 2024-08-18T13:43:19.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/34' 2024-08-18T13:43:19.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/38' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/11' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/16' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/32' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/30' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/14' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/18' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/13' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/33' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/17' 2024-08-18T13:43:20.386 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/35' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './167/37' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './167' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/19' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/15' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/31' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/36' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/20' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/12' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/39' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/10' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/34' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/38' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/11' 2024-08-18T13:43:20.387 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/16' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/32' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/30' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/14' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/18' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/13' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/33' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/17' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/35' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './168/37' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './168' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/19' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/15' 2024-08-18T13:43:20.388 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/31' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/36' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/20' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/12' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/39' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/10' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/34' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/38' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/11' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/16' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/32' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/30' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/14' 2024-08-18T13:43:20.389 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/18' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/13' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/33' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/17' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/35' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './169/37' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './169' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/19' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/15' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/31' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/36' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/20' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/12' 2024-08-18T13:43:20.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/39' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/10' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/34' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/38' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/11' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/16' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/32' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/30' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/14' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/18' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/13' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/33' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/17' 2024-08-18T13:43:20.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/35' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './170/37' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './170' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/19' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/15' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/31' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/36' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/20' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/12' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/39' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/10' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/34' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/38' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/11' 2024-08-18T13:43:20.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/16' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/32' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/30' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/14' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/18' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/13' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/33' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/17' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/35' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './171/37' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './171' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/19' 2024-08-18T13:43:20.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/15' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/31' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/36' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/20' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/12' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/39' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/10' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/34' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/38' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/11' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/16' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/32' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/30' 2024-08-18T13:43:20.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/14' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/18' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/13' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/33' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/17' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/35' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './172/37' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './172' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/19' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/15' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/31' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/36' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/20' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/12' 2024-08-18T13:43:20.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/39' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/10' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/34' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/38' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/11' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/16' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/32' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/30' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/14' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/18' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/13' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/33' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/17' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/35' 2024-08-18T13:43:20.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './173/37' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './173' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/19' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/15' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/31' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/36' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/20' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/12' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/39' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/10' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/34' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/38' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/11' 2024-08-18T13:43:20.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/16' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/32' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/30' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/14' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/18' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/13' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/33' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/17' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/35' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './174/37' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './174' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/19' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/15' 2024-08-18T13:43:20.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/31' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/36' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/20' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/12' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/39' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/10' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/34' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/38' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/11' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/16' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/32' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/30' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/14' 2024-08-18T13:43:20.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/18' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/13' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/33' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/17' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/35' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './175/37' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './175' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/19' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/15' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/31' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/36' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/20' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/12' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/39' 2024-08-18T13:43:20.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/10' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/34' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/38' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/11' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/16' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/32' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/30' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/14' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/18' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/13' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/33' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/17' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/35' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './176/37' 2024-08-18T13:43:20.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './176' 2024-08-18T13:43:20.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/19' 2024-08-18T13:43:20.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/15' 2024-08-18T13:43:21.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/31' 2024-08-18T13:43:21.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/36' 2024-08-18T13:43:21.390 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/20' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/12' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/39' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/10' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/34' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/38' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/11' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/16' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/32' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/30' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/14' 2024-08-18T13:43:21.391 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/18' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/13' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/33' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/17' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/35' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './177/37' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './177' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/19' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/15' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/31' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/36' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/20' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/12' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/39' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/10' 2024-08-18T13:43:21.392 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/34' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/38' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/11' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/16' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/32' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/30' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/14' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/18' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/13' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/33' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/17' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/35' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './178/37' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './178' 2024-08-18T13:43:21.393 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/19' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/15' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/31' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/36' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/20' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/12' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/39' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/10' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/34' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/38' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/11' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/16' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/32' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/30' 2024-08-18T13:43:21.394 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/14' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/18' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/13' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/33' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/17' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/35' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './179/37' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './179' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/19' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/15' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/31' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/36' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/20' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/12' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/39' 2024-08-18T13:43:21.395 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/10' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/34' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/38' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/11' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/16' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/32' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/30' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/14' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/18' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/13' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/33' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/17' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/35' 2024-08-18T13:43:21.396 INFO:tasks.workunit.client.0.smithi118.stdout:removed './180/37' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './180' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/19' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/15' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/31' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/36' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/20' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/12' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/39' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/10' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/34' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/38' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/11' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/16' 2024-08-18T13:43:21.397 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/32' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/30' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/14' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/18' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/13' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/33' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/17' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/35' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './181/37' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './181' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/19' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/15' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/31' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/36' 2024-08-18T13:43:21.398 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/20' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/12' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/39' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/10' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/34' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/38' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/11' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/16' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/32' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/30' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/14' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/18' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/13' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/33' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/17' 2024-08-18T13:43:21.399 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/35' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './182/37' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './182' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/19' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/15' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/31' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/36' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/20' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/12' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/39' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/10' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/34' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/38' 2024-08-18T13:43:21.400 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/11' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/16' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/32' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/30' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/14' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/18' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/13' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/33' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/17' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/35' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './183/37' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './183' 2024-08-18T13:43:21.401 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/19' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/15' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/31' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/36' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/20' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/12' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/39' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/10' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/34' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/38' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/11' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/16' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/32' 2024-08-18T13:43:21.402 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/30' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/14' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/18' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/13' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/33' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/17' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/35' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './184/37' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './184' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/19' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/15' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/31' 2024-08-18T13:43:21.403 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/36' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/20' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/12' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/39' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/10' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/34' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/38' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/11' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/16' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/32' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/30' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/14' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/18' 2024-08-18T13:43:21.404 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/13' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/33' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/17' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/35' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './185/37' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './185' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/19' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/15' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/31' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/36' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/20' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/12' 2024-08-18T13:43:21.405 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/39' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/10' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/34' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/38' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/11' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/16' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/32' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/30' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/14' 2024-08-18T13:43:21.406 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/18' 2024-08-18T13:43:22.594 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/13' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/33' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/17' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/35' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './186/37' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './186' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/19' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/15' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/31' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/36' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/20' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/12' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/39' 2024-08-18T13:43:22.595 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/10' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/34' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/38' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/11' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/16' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/32' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/30' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/14' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/18' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/13' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/33' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/17' 2024-08-18T13:43:22.596 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/35' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './187/37' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './187' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/19' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/15' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/31' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/36' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/20' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/12' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/39' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/10' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/34' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/38' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/11' 2024-08-18T13:43:22.597 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/16' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/32' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/30' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/14' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/18' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/13' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/33' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/17' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/35' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './188/37' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './188' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/19' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/15' 2024-08-18T13:43:22.598 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/31' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/36' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/20' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/12' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/39' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/10' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/34' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/38' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/11' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/16' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/32' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/30' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/14' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/18' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/13' 2024-08-18T13:43:22.599 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/33' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/17' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/35' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './189/37' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './189' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/19' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/15' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/31' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/36' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/20' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/12' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/39' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/10' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/34' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/38' 2024-08-18T13:43:22.600 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/11' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/16' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/32' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/30' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/14' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/18' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/13' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/33' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/17' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/35' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './190/37' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './190' 2024-08-18T13:43:22.601 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/19' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/15' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/31' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/36' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/20' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/12' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/39' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/10' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/34' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/38' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/11' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/16' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/32' 2024-08-18T13:43:22.602 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/30' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/14' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/18' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/13' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/33' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/17' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/35' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './191/37' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './191' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/19' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/15' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/31' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/36' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/20' 2024-08-18T13:43:22.603 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/12' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/39' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/10' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/34' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/38' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/11' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/16' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/32' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/30' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/14' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/18' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/13' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/33' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/17' 2024-08-18T13:43:22.604 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/35' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './192/37' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './192' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/19' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/15' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/31' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/36' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/20' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/12' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/39' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/10' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/34' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/38' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/11' 2024-08-18T13:43:22.605 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/16' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/32' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/30' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/14' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/18' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/13' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/33' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/17' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/35' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './193/37' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './193' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/19' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/15' 2024-08-18T13:43:22.606 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/31' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/36' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/20' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/12' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/39' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/10' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/34' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/38' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/11' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/16' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/32' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/30' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/14' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/18' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/13' 2024-08-18T13:43:22.607 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/33' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/17' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/35' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './194/37' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './194' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/19' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/15' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/31' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/36' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/20' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/12' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/39' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/10' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/34' 2024-08-18T13:43:22.608 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/38' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/11' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/16' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/32' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/30' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/14' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/18' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/13' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/33' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/17' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/35' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './195/37' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './195' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/19' 2024-08-18T13:43:22.609 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/15' 2024-08-18T13:43:22.610 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/31' 2024-08-18T13:43:22.610 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/36' 2024-08-18T13:43:22.610 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/20' 2024-08-18T13:43:22.610 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/12' 2024-08-18T13:43:22.610 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/39' 2024-08-18T13:43:22.610 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/10' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/34' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/38' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/11' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/16' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/32' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/30' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/14' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/18' 2024-08-18T13:43:23.644 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/13' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/33' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/17' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/35' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './196/37' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './196' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/19' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/15' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/31' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/36' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/20' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/12' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/39' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/10' 2024-08-18T13:43:23.645 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/34' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/38' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/11' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/16' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/32' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/30' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/14' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/18' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/13' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/33' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/17' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/35' 2024-08-18T13:43:23.646 INFO:tasks.workunit.client.0.smithi118.stdout:removed './197/37' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './197' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/19' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/15' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/31' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/36' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/20' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/12' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/39' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/10' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/34' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/38' 2024-08-18T13:43:23.647 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/11' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/16' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/32' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/30' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/14' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/18' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/13' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/33' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/17' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/35' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './198/37' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './198' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/19' 2024-08-18T13:43:23.648 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/15' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/31' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/36' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/20' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/12' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/39' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/10' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/34' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/38' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/11' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/16' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/32' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/30' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/14' 2024-08-18T13:43:23.649 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/18' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/13' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/33' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/17' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/35' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './199/37' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './199' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/21' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/27' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/31' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/36' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/20' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/39' 2024-08-18T13:43:23.650 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/34' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/38' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/28' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/24' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/29' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/32' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/22' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/25' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/30' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/33' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/23' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/35' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/26' 2024-08-18T13:43:23.651 INFO:tasks.workunit.client.0.smithi118.stdout:removed './200/37' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './200' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/21' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/27' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/31' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/36' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/20' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/39' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/34' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/38' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/28' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/24' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/29' 2024-08-18T13:43:23.652 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/32' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/22' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/25' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/30' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/33' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/23' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/35' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/26' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './201/37' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './201' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/21' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/27' 2024-08-18T13:43:23.653 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/31' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/36' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/20' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/39' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/34' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/38' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/28' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/24' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/29' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/32' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/22' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/25' 2024-08-18T13:43:23.654 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/30' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/33' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/23' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/35' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/26' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './202/37' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './202' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/21' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/27' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/31' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/36' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/20' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/39' 2024-08-18T13:43:23.655 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/34' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/38' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/28' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/24' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/29' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/32' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/22' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/25' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/30' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/33' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/23' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/35' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/26' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed './203/37' 2024-08-18T13:43:23.656 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './203' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/21' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/27' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/31' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/36' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/20' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/39' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/34' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/38' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/28' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/24' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/29' 2024-08-18T13:43:23.657 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/32' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/22' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/25' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/30' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/33' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/23' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/35' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/26' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './204/37' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './204' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/21' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/27' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/31' 2024-08-18T13:43:23.658 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/36' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/20' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/39' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/34' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/38' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/28' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/24' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/29' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/32' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/22' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/25' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/30' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/33' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/23' 2024-08-18T13:43:23.659 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/35' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/26' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './205/37' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './205' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/21' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/27' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/31' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/36' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/20' 2024-08-18T13:43:23.660 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/39' 2024-08-18T13:43:24.724 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/34' 2024-08-18T13:43:24.724 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/38' 2024-08-18T13:43:24.724 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/28' 2024-08-18T13:43:24.724 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/24' 2024-08-18T13:43:24.724 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/29' 2024-08-18T13:43:24.724 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/32' 2024-08-18T13:43:24.724 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/22' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/25' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/30' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/33' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/23' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/35' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/26' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './206/37' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './206' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/21' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/27' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/31' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/36' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/20' 2024-08-18T13:43:24.725 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/39' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/34' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/38' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/28' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/24' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/29' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/32' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/22' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/25' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/30' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/33' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/23' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/35' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/26' 2024-08-18T13:43:24.726 INFO:tasks.workunit.client.0.smithi118.stdout:removed './207/37' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './207' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/21' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/27' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/31' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/36' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/20' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/39' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/34' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/38' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/28' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/24' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/29' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/32' 2024-08-18T13:43:24.727 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/22' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/25' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/30' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/33' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/23' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/35' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/26' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './208/37' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './208' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/21' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/27' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/31' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/36' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/20' 2024-08-18T13:43:24.728 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/39' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/34' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/38' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/28' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/24' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/29' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/32' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/22' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/25' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/30' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/33' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/23' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/35' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/26' 2024-08-18T13:43:24.729 INFO:tasks.workunit.client.0.smithi118.stdout:removed './209/37' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './209' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/21' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/27' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/31' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/36' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/20' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/39' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/34' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/38' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/28' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/24' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/29' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/32' 2024-08-18T13:43:24.730 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/22' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/25' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/30' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/33' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/23' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/35' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/26' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './210/37' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './210' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/21' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/27' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/31' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/36' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/20' 2024-08-18T13:43:24.731 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/39' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/34' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/38' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/28' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/24' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/29' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/32' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/22' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/25' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/30' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/33' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/23' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/35' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/26' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed './211/37' 2024-08-18T13:43:24.732 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './211' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/21' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/27' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/31' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/36' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/20' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/39' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/34' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/38' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/28' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/24' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/29' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/32' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/22' 2024-08-18T13:43:24.733 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/25' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/30' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/33' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/23' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/35' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/26' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './212/37' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './212' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/21' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/27' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/31' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/36' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/20' 2024-08-18T13:43:24.734 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/39' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/34' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/38' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/28' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/24' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/29' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/32' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/22' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/25' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/30' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/33' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/23' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/35' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/26' 2024-08-18T13:43:24.735 INFO:tasks.workunit.client.0.smithi118.stdout:removed './213/37' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './213' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/21' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/27' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/31' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/36' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/20' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/39' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/34' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/38' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/28' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/24' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/29' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/32' 2024-08-18T13:43:24.736 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/22' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/25' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/30' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/33' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/23' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/35' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/26' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './214/37' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './214' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/21' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/27' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/31' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/36' 2024-08-18T13:43:24.737 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/20' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/39' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/34' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/38' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/28' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/24' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/29' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/32' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/22' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/25' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/30' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/33' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/23' 2024-08-18T13:43:24.738 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/35' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/26' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './215/37' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './215' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/21' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/27' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/31' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/36' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/20' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/39' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/34' 2024-08-18T13:43:24.739 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/38' 2024-08-18T13:43:25.842 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/28' 2024-08-18T13:43:25.842 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/24' 2024-08-18T13:43:25.842 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/29' 2024-08-18T13:43:25.842 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/32' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/22' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/25' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/30' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/33' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/23' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/35' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/26' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './216/37' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './216' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/21' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/27' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/31' 2024-08-18T13:43:25.843 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/36' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/20' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/39' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/34' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/38' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/28' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/24' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/29' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/32' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/22' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/25' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/30' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/33' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/23' 2024-08-18T13:43:25.844 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/35' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/26' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './217/37' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './217' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/21' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/27' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/31' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/36' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/20' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/39' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/34' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/38' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/28' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/24' 2024-08-18T13:43:25.845 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/29' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/32' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/22' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/25' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/30' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/33' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/23' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/35' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/26' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './218/37' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './218' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/21' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/27' 2024-08-18T13:43:25.846 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/31' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/36' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/20' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/39' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/34' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/38' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/28' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/24' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/29' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/32' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/22' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/25' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/30' 2024-08-18T13:43:25.847 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/33' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/23' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/35' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/26' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './219/37' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './219' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/21' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/27' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/31' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/36' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/20' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/39' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/34' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/38' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/28' 2024-08-18T13:43:25.848 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/24' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/29' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/32' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/22' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/25' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/30' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/33' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/23' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/35' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/26' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './220/37' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './220' 2024-08-18T13:43:25.849 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/21' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/27' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/31' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/36' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/20' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/39' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/34' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/38' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/28' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/24' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/29' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/32' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/22' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/25' 2024-08-18T13:43:25.850 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/30' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/33' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/23' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/35' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/26' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './221/37' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './221' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/21' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/27' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/31' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/36' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/20' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/39' 2024-08-18T13:43:25.851 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/34' 2024-08-18T13:43:25.852 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/38' 2024-08-18T13:43:25.852 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/28' 2024-08-18T13:43:25.852 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/24' 2024-08-18T13:43:25.852 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/29' 2024-08-18T13:43:25.852 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/32' 2024-08-18T13:43:25.852 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/22' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/25' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/30' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/33' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/23' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/35' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/26' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './222/37' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './222' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/21' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/27' 2024-08-18T13:43:25.854 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/31' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/36' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/20' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/39' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/34' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/38' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/28' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/24' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/29' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/32' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/22' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/25' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/30' 2024-08-18T13:43:25.855 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/33' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/23' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/35' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/26' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './223/37' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './223' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/21' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/27' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/31' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/36' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/20' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/39' 2024-08-18T13:43:25.856 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/34' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/38' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/28' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/24' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/29' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/32' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/22' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/25' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/30' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/33' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/23' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/35' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/26' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed './224/37' 2024-08-18T13:43:25.857 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './224' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/21' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/27' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/31' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/36' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/20' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/39' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/34' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/38' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/28' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/24' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/29' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/32' 2024-08-18T13:43:25.858 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/22' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/25' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/30' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/33' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/23' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/35' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/26' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './225/37' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './225' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/21' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/27' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/31' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/36' 2024-08-18T13:43:25.859 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/20' 2024-08-18T13:43:25.860 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/39' 2024-08-18T13:43:25.860 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/34' 2024-08-18T13:43:25.860 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/38' 2024-08-18T13:43:25.860 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/28' 2024-08-18T13:43:25.860 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/24' 2024-08-18T13:43:26.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/29' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/32' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/22' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/25' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/30' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/33' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/23' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/35' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/26' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './226/37' 2024-08-18T13:43:26.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './226' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/21' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/27' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/31' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/36' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/20' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/39' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/34' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/38' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/28' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/24' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/29' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/32' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/22' 2024-08-18T13:43:26.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/25' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/30' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/33' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/23' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/35' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/26' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './227/37' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './227' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/21' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/27' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/31' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/36' 2024-08-18T13:43:26.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/20' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/39' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/34' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/38' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/28' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/24' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/29' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/32' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/22' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/25' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/30' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/33' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/23' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/35' 2024-08-18T13:43:26.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/26' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './228/37' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './228' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/21' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/27' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/31' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/36' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/20' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/39' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/34' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/38' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/28' 2024-08-18T13:43:26.912 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/24' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/29' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/32' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/22' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/25' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/30' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/33' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/23' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/35' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/26' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './229/37' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './229' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/21' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/27' 2024-08-18T13:43:26.913 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/31' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/36' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/20' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/39' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/34' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/38' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/28' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/24' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/29' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/32' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/22' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/25' 2024-08-18T13:43:26.914 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/30' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/33' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/23' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/35' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/26' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './230/37' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './230' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/21' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/27' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/31' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/36' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/20' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/39' 2024-08-18T13:43:26.915 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/34' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/38' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/28' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/24' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/29' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/32' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/22' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/25' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/30' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/33' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/23' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/35' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/26' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed './231/37' 2024-08-18T13:43:26.916 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './231' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/21' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/27' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/31' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/36' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/20' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/39' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/34' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/38' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/28' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/24' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/29' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/32' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/22' 2024-08-18T13:43:26.917 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/25' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/30' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/33' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/23' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/35' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/26' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './232/37' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './232' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/21' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/27' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/31' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/36' 2024-08-18T13:43:26.918 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/20' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/39' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/34' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/38' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/28' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/24' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/29' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/32' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/22' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/25' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/30' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/33' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/23' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/35' 2024-08-18T13:43:26.919 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/26' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './233/37' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './233' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/21' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/27' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/31' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/36' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/20' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/39' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/34' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/38' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/28' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/24' 2024-08-18T13:43:26.920 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/29' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/32' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/22' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/25' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/30' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/33' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/23' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/35' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/26' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './234/37' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './234' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/21' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/27' 2024-08-18T13:43:26.921 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/31' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/36' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/20' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/39' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/34' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/38' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/28' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/24' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/29' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/32' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/22' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/25' 2024-08-18T13:43:26.922 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/30' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/33' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/23' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/35' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/26' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './235/37' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './235' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/21' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/27' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/31' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/36' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/20' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/39' 2024-08-18T13:43:26.923 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/34' 2024-08-18T13:43:26.924 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/38' 2024-08-18T13:43:26.924 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/28' 2024-08-18T13:43:26.924 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/24' 2024-08-18T13:43:26.924 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/29' 2024-08-18T13:43:26.924 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/32' 2024-08-18T13:43:28.105 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/22' 2024-08-18T13:43:28.105 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/25' 2024-08-18T13:43:28.105 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/30' 2024-08-18T13:43:28.105 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/33' 2024-08-18T13:43:28.105 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/23' 2024-08-18T13:43:28.105 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/35' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/26' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './236/37' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './236' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/21' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/27' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/31' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/36' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/20' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/39' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/34' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/38' 2024-08-18T13:43:28.106 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/28' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/24' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/29' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/32' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/22' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/25' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/30' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/33' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/23' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/35' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/26' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './237/37' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './237' 2024-08-18T13:43:28.107 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/21' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/27' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/31' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/36' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/20' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/39' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/34' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/38' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/28' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/24' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/29' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/32' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/22' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/25' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/30' 2024-08-18T13:43:28.108 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/33' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/23' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/35' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/26' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './238/37' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './238' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/21' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/27' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/31' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/36' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/20' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/39' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/34' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/38' 2024-08-18T13:43:28.109 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/28' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/24' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/29' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/32' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/22' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/25' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/30' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/33' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/23' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/35' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/26' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './239/37' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './239' 2024-08-18T13:43:28.110 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/21' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/27' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/31' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/36' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/20' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/39' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/34' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/38' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/28' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/24' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/29' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/32' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/22' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/25' 2024-08-18T13:43:28.111 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/30' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/33' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/23' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/35' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/26' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './240/37' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './240' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/21' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/27' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/31' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/36' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/20' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/39' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/34' 2024-08-18T13:43:28.112 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/38' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/28' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/24' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/29' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/32' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/22' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/25' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/30' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/33' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/23' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/35' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/26' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './241/37' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './241' 2024-08-18T13:43:28.113 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/21' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/27' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/31' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/36' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/20' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/39' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/34' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/38' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/28' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/24' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/29' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/32' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/22' 2024-08-18T13:43:28.114 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/25' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/30' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/33' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/23' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/35' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/26' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './242/37' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './242' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/21' 2024-08-18T13:43:28.115 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/27' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/31' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/36' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/20' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/39' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/34' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/38' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/28' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/24' 2024-08-18T13:43:28.116 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/29' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/32' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/22' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/25' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/30' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/33' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/23' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/35' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/26' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './243/37' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './243' 2024-08-18T13:43:28.117 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/21' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/27' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/31' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/36' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/20' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/39' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/34' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/38' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/28' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/24' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/29' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/32' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/22' 2024-08-18T13:43:28.118 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/25' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/30' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/33' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/23' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/35' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/26' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './244/37' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './244' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/21' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/27' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/31' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/36' 2024-08-18T13:43:28.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/20' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/39' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/34' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/38' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/28' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/24' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/29' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/32' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/22' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/25' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/30' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/33' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/23' 2024-08-18T13:43:28.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/35' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/26' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './245/37' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './245' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/21' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/27' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/31' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/36' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/20' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/39' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/34' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/38' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/28' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/24' 2024-08-18T13:43:28.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/29' 2024-08-18T13:43:28.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/32' 2024-08-18T13:43:28.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/22' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/25' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/30' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/33' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/23' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/35' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/26' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './246/37' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './246' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/21' 2024-08-18T13:43:29.215 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/27' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/31' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/36' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/20' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/39' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/34' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/38' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/28' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/24' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/29' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/32' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/22' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/25' 2024-08-18T13:43:29.216 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/30' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/33' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/23' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/35' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/26' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './247/37' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './247' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/21' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/27' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/31' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/36' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/20' 2024-08-18T13:43:29.217 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/39' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/34' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/38' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/28' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/24' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/29' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/32' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/22' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/25' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/30' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/33' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/23' 2024-08-18T13:43:29.218 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/35' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/26' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './248/37' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './248' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/21' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/27' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/31' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/36' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/20' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/39' 2024-08-18T13:43:29.219 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/34' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/38' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/28' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/24' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/29' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/32' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/22' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/25' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/30' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/33' 2024-08-18T13:43:29.220 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/23' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/35' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/26' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './249/37' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './249' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/21' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/27' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/31' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/36' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/20' 2024-08-18T13:43:29.221 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/39' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/34' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/38' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/28' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/24' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/29' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/32' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/22' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/25' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/30' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/33' 2024-08-18T13:43:29.222 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/23' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/35' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/26' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './250/37' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './250' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/21' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/27' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/31' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/36' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/20' 2024-08-18T13:43:29.223 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/39' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/34' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/38' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/28' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/24' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/29' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/32' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/22' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/25' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/30' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/33' 2024-08-18T13:43:29.224 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/23' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/35' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/26' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './251/37' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './251' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/21' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/27' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/31' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/36' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/20' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/39' 2024-08-18T13:43:29.225 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/34' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/38' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/28' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/24' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/29' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/32' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/22' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/25' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/30' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/33' 2024-08-18T13:43:29.226 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/23' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/35' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/26' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './252/37' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './252' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/21' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/27' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/31' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/36' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/20' 2024-08-18T13:43:29.227 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/39' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/34' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/38' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/28' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/24' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/29' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/32' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/22' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/25' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/30' 2024-08-18T13:43:29.228 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/33' 2024-08-18T13:43:29.229 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/23' 2024-08-18T13:43:29.229 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/35' 2024-08-18T13:43:29.229 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/26' 2024-08-18T13:43:29.229 INFO:tasks.workunit.client.0.smithi118.stdout:removed './253/37' 2024-08-18T13:43:29.229 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './253' 2024-08-18T13:43:29.229 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/21' 2024-08-18T13:43:29.229 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/27' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/31' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/36' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/20' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/39' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/34' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/38' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/28' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/24' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/29' 2024-08-18T13:43:29.230 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/32' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/22' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/25' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/30' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/33' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/23' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/35' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/26' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed './254/37' 2024-08-18T13:43:29.231 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './254' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/21' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/27' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/31' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/36' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/20' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/39' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/34' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/38' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/28' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/24' 2024-08-18T13:43:29.232 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/29' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/32' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/22' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/25' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/30' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/33' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/23' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/35' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/26' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './255/37' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './255' 2024-08-18T13:43:29.233 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/21' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/27' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/31' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/36' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/20' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/39' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/34' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/38' 2024-08-18T13:43:29.234 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/28' 2024-08-18T13:43:29.235 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/24' 2024-08-18T13:43:29.235 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/29' 2024-08-18T13:43:29.235 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/32' 2024-08-18T13:43:29.235 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/22' 2024-08-18T13:43:29.235 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/25' 2024-08-18T13:43:29.235 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/30' 2024-08-18T13:43:30.251 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/33' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/23' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/35' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/26' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './256/37' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './256' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/21' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/27' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/31' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/36' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/20' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/39' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/34' 2024-08-18T13:43:30.252 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/38' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/28' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/24' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/29' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/32' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/22' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/25' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/30' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/33' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/23' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/35' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/26' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './257/37' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './257' 2024-08-18T13:43:30.253 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/21' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/27' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/31' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/36' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/20' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/39' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/34' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/38' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/28' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/24' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/29' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/32' 2024-08-18T13:43:30.254 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/22' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/25' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/30' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/33' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/23' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/35' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/26' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './258/37' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './258' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/21' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/27' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/31' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/36' 2024-08-18T13:43:30.255 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/20' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/39' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/34' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/38' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/28' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/24' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/29' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/32' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/22' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/25' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/30' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/33' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/23' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/35' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/26' 2024-08-18T13:43:30.256 INFO:tasks.workunit.client.0.smithi118.stdout:removed './259/37' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './259' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/21' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/27' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/31' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/36' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/20' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/39' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/34' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/38' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/28' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/24' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/29' 2024-08-18T13:43:30.257 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/32' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/22' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/25' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/30' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/33' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/23' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/35' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/26' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './260/37' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './260' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/21' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/27' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/31' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/36' 2024-08-18T13:43:30.258 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/20' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/39' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/34' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/38' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/28' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/24' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/29' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/32' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/22' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/25' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/30' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/33' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/23' 2024-08-18T13:43:30.259 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/35' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/26' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './261/37' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './261' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/21' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/27' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/31' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/36' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/20' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/39' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/34' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/38' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/28' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/24' 2024-08-18T13:43:30.260 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/29' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/32' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/22' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/25' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/30' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/33' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/23' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/35' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/26' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './262/37' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './262' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/21' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/27' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/31' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/36' 2024-08-18T13:43:30.261 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/20' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/39' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/34' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/38' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/28' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/24' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/29' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/32' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/22' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/25' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/30' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/33' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/23' 2024-08-18T13:43:30.262 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/35' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/26' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './263/37' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './263' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/21' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/27' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/31' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/36' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/20' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/39' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/34' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/38' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/28' 2024-08-18T13:43:30.263 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/24' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/29' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/32' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/22' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/25' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/30' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/33' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/23' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/35' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/26' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './264/37' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './264' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/21' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/27' 2024-08-18T13:43:30.264 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/31' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/36' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/20' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/39' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/34' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/38' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/28' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/24' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/29' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/32' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/22' 2024-08-18T13:43:30.265 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/25' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/30' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/33' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/23' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/35' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/26' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './265/37' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './265' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/21' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/27' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/31' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/36' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/20' 2024-08-18T13:43:30.266 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/39' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/34' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/38' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/28' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/24' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/29' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/32' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/22' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/25' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/30' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/33' 2024-08-18T13:43:30.267 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/23' 2024-08-18T13:43:31.299 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/35' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/26' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './266/37' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './266' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/21' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/27' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/31' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/36' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/20' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/39' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/34' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/38' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/28' 2024-08-18T13:43:31.300 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/24' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/29' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/32' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/22' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/25' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/30' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/33' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/23' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/35' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/26' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './267/37' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './267' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/21' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/27' 2024-08-18T13:43:31.301 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/31' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/36' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/20' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/39' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/34' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/38' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/28' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/24' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/29' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/32' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/22' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/25' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/30' 2024-08-18T13:43:31.302 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/33' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/23' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/35' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/26' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './268/37' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './268' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/21' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/27' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/31' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/36' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/20' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/39' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/34' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/38' 2024-08-18T13:43:31.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/28' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/24' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/29' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/32' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/22' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/25' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/30' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/33' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/23' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/35' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/26' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './269/37' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './269' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/21' 2024-08-18T13:43:31.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/27' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/31' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/36' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/20' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/39' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/34' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/38' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/28' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/24' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/29' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/32' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/22' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/25' 2024-08-18T13:43:31.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/30' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/33' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/23' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/35' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/26' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './270/37' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './270' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/21' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/27' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/31' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/36' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/20' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/39' 2024-08-18T13:43:31.306 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/34' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/38' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/28' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/24' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/29' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/32' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/22' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/25' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/30' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/33' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/23' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/35' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/26' 2024-08-18T13:43:31.307 INFO:tasks.workunit.client.0.smithi118.stdout:removed './271/37' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './271' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/21' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/27' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/31' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/36' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/20' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/39' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/34' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/38' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/28' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/24' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/29' 2024-08-18T13:43:31.308 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/32' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/22' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/25' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/30' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/33' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/23' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/35' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/26' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './272/37' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './272' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/21' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/27' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/31' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/36' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/20' 2024-08-18T13:43:31.309 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/39' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/34' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/38' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/28' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/24' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/29' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/32' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/22' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/25' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/30' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/33' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/23' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/35' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/26' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed './273/37' 2024-08-18T13:43:31.310 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './273' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/21' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/27' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/31' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/36' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/20' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/39' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/34' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/38' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/28' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/24' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/29' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/32' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/22' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/25' 2024-08-18T13:43:31.311 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/30' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/33' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/23' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/35' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/26' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './274/37' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './274' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/21' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/27' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/31' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/36' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/20' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/39' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/34' 2024-08-18T13:43:31.312 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/38' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/28' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/24' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/29' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/32' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/22' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/25' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/30' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/33' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/23' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/35' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/26' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './275/37' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './275' 2024-08-18T13:43:31.313 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/21' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/27' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/31' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/36' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/20' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/39' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/34' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/38' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/28' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/24' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/29' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/32' 2024-08-18T13:43:31.314 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/22' 2024-08-18T13:43:31.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/25' 2024-08-18T13:43:31.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/30' 2024-08-18T13:43:31.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/33' 2024-08-18T13:43:31.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/23' 2024-08-18T13:43:31.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/35' 2024-08-18T13:43:31.315 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/26' 2024-08-18T13:43:32.441 INFO:tasks.workunit.client.0.smithi118.stdout:removed './276/37' 2024-08-18T13:43:32.441 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './276' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/21' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/27' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/31' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/36' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/20' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/39' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/34' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/38' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/28' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/24' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/29' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/32' 2024-08-18T13:43:32.442 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/22' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/25' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/30' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/33' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/23' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/35' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/26' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './277/37' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './277' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/21' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/27' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/31' 2024-08-18T13:43:32.443 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/36' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/20' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/39' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/34' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/38' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/28' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/24' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/29' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/32' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/22' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/25' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/30' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/33' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/23' 2024-08-18T13:43:32.444 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/35' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/26' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './278/37' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './278' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/21' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/27' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/31' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/36' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/20' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/39' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/34' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/38' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/28' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/24' 2024-08-18T13:43:32.445 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/29' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/32' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/22' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/25' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/30' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/33' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/23' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/35' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/26' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './279/37' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './279' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/21' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/27' 2024-08-18T13:43:32.446 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/31' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/36' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/20' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/39' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/34' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/38' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/28' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/24' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/29' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/32' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/22' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/25' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/30' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/33' 2024-08-18T13:43:32.447 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/23' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/35' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/26' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './280/37' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './280' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/21' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/27' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/31' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/36' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/20' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/39' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/34' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/38' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/28' 2024-08-18T13:43:32.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/24' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/29' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/32' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/22' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/25' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/30' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/33' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/23' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/35' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/26' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './281/37' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './281' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/21' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/27' 2024-08-18T13:43:32.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/31' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/36' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/20' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/39' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/34' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/38' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/28' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/24' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/29' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/32' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/22' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/25' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/30' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/33' 2024-08-18T13:43:32.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/23' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/35' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/26' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './282/37' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './282' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/21' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/27' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/31' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/36' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/20' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/39' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/34' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/38' 2024-08-18T13:43:32.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/28' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/24' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/29' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/32' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/22' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/25' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/30' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/33' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/23' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/35' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/26' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './283/37' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './283' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/21' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/27' 2024-08-18T13:43:32.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/31' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/36' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/20' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/39' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/34' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/38' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/28' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/24' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/29' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/32' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/22' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/25' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/30' 2024-08-18T13:43:32.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/33' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/23' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/35' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/26' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './284/37' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './284' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/21' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/27' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/31' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/36' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/20' 2024-08-18T13:43:32.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/39' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/34' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/38' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/28' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/24' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/29' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/32' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/22' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/25' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/30' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/33' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/23' 2024-08-18T13:43:32.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/35' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/26' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './285/37' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './285' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/21' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/27' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/31' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/36' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/20' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/39' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/34' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/38' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/28' 2024-08-18T13:43:32.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/24' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/29' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/32' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/22' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/25' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/30' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/33' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/23' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/35' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/26' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed './286/37' 2024-08-18T13:43:32.457 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './286' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/21' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/27' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/31' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/36' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/20' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/39' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/34' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/38' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/28' 2024-08-18T13:43:33.538 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/24' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/29' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/32' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/22' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/25' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/30' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/33' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/23' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/35' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/26' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './287/37' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './287' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/21' 2024-08-18T13:43:33.539 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/27' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/31' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/36' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/20' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/39' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/34' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/38' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/28' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/24' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/29' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/32' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/22' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/25' 2024-08-18T13:43:33.540 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/30' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/33' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/23' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/35' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/26' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './288/37' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './288' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/21' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/27' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/31' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/36' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/20' 2024-08-18T13:43:33.541 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/39' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/34' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/38' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/28' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/24' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/29' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/32' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/22' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/25' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/30' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/33' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/23' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/35' 2024-08-18T13:43:33.542 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/26' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './289/37' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './289' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/21' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/27' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/31' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/36' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/20' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/39' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/34' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/38' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/28' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/24' 2024-08-18T13:43:33.543 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/29' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/32' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/22' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/25' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/30' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/33' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/23' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/35' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/26' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './290/37' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './290' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/21' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/27' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/31' 2024-08-18T13:43:33.544 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/36' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/20' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/39' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/34' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/38' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/28' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/24' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/29' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/32' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/22' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/25' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/30' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/33' 2024-08-18T13:43:33.545 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/23' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/35' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/26' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './291/37' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './291' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/21' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/27' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/31' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/36' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/20' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/39' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/34' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/38' 2024-08-18T13:43:33.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/28' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/24' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/29' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/32' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/22' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/25' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/30' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/33' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/23' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/35' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/26' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './292/37' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './292' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/21' 2024-08-18T13:43:33.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/27' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/31' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/36' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/20' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/39' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/34' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/38' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/28' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/24' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/29' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/32' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/22' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/25' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/30' 2024-08-18T13:43:33.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/33' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/23' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/35' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/26' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './293/37' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './293' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/21' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/27' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/31' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/36' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/20' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/39' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/34' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/38' 2024-08-18T13:43:33.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/28' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/24' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/29' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/32' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/22' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/25' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/30' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/33' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/23' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/35' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/26' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './294/37' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './294' 2024-08-18T13:43:33.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/21' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/27' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/31' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/36' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/20' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/39' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/34' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/38' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/28' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/24' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/29' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/32' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/22' 2024-08-18T13:43:33.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/25' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/30' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/33' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/23' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/35' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/26' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './295/37' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './295' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/21' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/27' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/31' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/36' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/20' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/39' 2024-08-18T13:43:33.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/34' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/38' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/28' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/24' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/29' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/32' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/22' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/25' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/30' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/33' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/23' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/35' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/26' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './296/37' 2024-08-18T13:43:33.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './296' 2024-08-18T13:43:33.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/21' 2024-08-18T13:43:33.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/27' 2024-08-18T13:43:34.572 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/31' 2024-08-18T13:43:34.572 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/36' 2024-08-18T13:43:34.572 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/20' 2024-08-18T13:43:34.572 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/39' 2024-08-18T13:43:34.572 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/34' 2024-08-18T13:43:34.572 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/38' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/28' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/24' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/29' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/32' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/22' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/25' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/30' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/33' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/23' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/35' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/26' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './297/37' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './297' 2024-08-18T13:43:34.573 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/21' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/27' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/31' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/36' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/20' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/39' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/34' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/38' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/28' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/24' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/29' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/32' 2024-08-18T13:43:34.574 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/22' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/25' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/30' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/33' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/23' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/35' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/26' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './298/37' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './298' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/21' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/27' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/31' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/36' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/20' 2024-08-18T13:43:34.575 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/39' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/34' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/38' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/28' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/24' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/29' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/32' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/22' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/25' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/30' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/33' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/23' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/35' 2024-08-18T13:43:34.576 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/26' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './299/37' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './299' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/19' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/15' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/20' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/12' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/10' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/11' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/16' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/14' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/18' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/13' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/166/17' 2024-08-18T13:43:34.577 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/166' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/19' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/15' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/20' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/12' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/10' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/11' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/16' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/14' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/18' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/13' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/147/17' 2024-08-18T13:43:34.578 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/147' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/19' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/15' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/20' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/12' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/10' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/11' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/16' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/14' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/18' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/13' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/151/17' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/151' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/19' 2024-08-18T13:43:34.579 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/15' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/20' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/12' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/10' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/11' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/16' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/14' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/18' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/13' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/170/17' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/170' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/19' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/15' 2024-08-18T13:43:34.580 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/20' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/12' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/10' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/11' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/16' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/14' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/18' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/13' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/167/17' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/167' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/19' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/15' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/20' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/12' 2024-08-18T13:43:34.581 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/10' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/11' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/16' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/14' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/18' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/13' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/140/17' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/140' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/19' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/15' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/20' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/12' 2024-08-18T13:43:34.582 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/10' 2024-08-18T13:43:35.272 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/11' 2024-08-18T13:43:35.272 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/16' 2024-08-18T13:43:35.272 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/14' 2024-08-18T13:43:35.272 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/18' 2024-08-18T13:43:35.272 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/13' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/142/17' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/142' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/19' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/15' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/20' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/12' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/10' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/11' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/16' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/14' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/18' 2024-08-18T13:43:35.273 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/13' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/150/17' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/150' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/19' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/15' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/20' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/12' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/10' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/11' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/16' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/14' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/18' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/13' 2024-08-18T13:43:35.274 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/105/17' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/105' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/19' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/15' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/20' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/12' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/10' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/11' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/16' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/14' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/18' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/13' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/149/17' 2024-08-18T13:43:35.275 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/149' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/19' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/15' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/20' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/12' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/10' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/11' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/16' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/14' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/18' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/13' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/109/17' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/109' 2024-08-18T13:43:35.276 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/19' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/15' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/20' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/12' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/10' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/11' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/16' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/14' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/18' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/13' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/158/17' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/158' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/19' 2024-08-18T13:43:35.277 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/15' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/20' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/12' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/10' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/11' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/16' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/14' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/18' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/13' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/106/17' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/106' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/19' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/15' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/20' 2024-08-18T13:43:35.278 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/12' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/10' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/11' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/16' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/14' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/18' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/13' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/189/17' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/189' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/19' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/15' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/20' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/12' 2024-08-18T13:43:35.279 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/10' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/11' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/16' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/14' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/18' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/13' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/133/17' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/133' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/19' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/15' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/20' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/12' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/10' 2024-08-18T13:43:35.280 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/11' 2024-08-18T13:43:35.903 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/16' 2024-08-18T13:43:35.903 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/14' 2024-08-18T13:43:35.903 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/18' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/13' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/173/17' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/173' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/19' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/15' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/20' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/12' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/10' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/11' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/16' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/14' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/18' 2024-08-18T13:43:35.904 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/13' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/174/17' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/174' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/19' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/15' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/20' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/12' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/10' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/11' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/16' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/14' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/18' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/13' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/155/17' 2024-08-18T13:43:35.905 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/155' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/19' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/15' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/20' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/12' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/10' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/11' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/16' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/14' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/18' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/13' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/114/17' 2024-08-18T13:43:35.906 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/114' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/19' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/15' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/20' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/12' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/10' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/11' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/16' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/14' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/18' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/13' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/159/17' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/159' 2024-08-18T13:43:35.907 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/19' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/15' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/20' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/12' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/10' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/11' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/16' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/14' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/18' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/13' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/187/17' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/187' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/19' 2024-08-18T13:43:35.908 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/15' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/20' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/12' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/10' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/11' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/16' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/14' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/18' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/13' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/185/17' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/185' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/19' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/15' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/20' 2024-08-18T13:43:35.909 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/12' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/10' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/11' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/16' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/14' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/18' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/13' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/120/17' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/120' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/19' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/15' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/20' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/12' 2024-08-18T13:43:35.910 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/10' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/11' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/16' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/14' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/18' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/13' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/110/17' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/110' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/19' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/15' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/20' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/12' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/10' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/11' 2024-08-18T13:43:35.911 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/16' 2024-08-18T13:43:36.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/14' 2024-08-18T13:43:36.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/18' 2024-08-18T13:43:36.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/13' 2024-08-18T13:43:36.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/137/17' 2024-08-18T13:43:36.546 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/137' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/19' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/15' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/20' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/12' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/10' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/11' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/16' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/14' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/18' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/13' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/198/17' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/198' 2024-08-18T13:43:36.547 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/19' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/15' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/20' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/12' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/10' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/11' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/16' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/14' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/18' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/13' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/136/17' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/136' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/19' 2024-08-18T13:43:36.548 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/15' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/20' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/12' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/10' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/11' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/16' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/14' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/18' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/13' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/125/17' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/125' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/19' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/15' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/20' 2024-08-18T13:43:36.549 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/12' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/10' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/11' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/16' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/14' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/18' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/13' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/194/17' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/194' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/19' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/15' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/20' 2024-08-18T13:43:36.550 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/12' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/10' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/11' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/16' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/14' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/18' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/13' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/119/17' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/119' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/19' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/15' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/20' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/12' 2024-08-18T13:43:36.551 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/10' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/11' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/16' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/14' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/18' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/13' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/157/17' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/157' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/19' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/15' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/20' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/12' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/10' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/11' 2024-08-18T13:43:36.552 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/16' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/14' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/18' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/13' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/172/17' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/172' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/19' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/15' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/20' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/12' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/10' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/11' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/16' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/14' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/18' 2024-08-18T13:43:36.553 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/13' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/186/17' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/186' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/19' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/15' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/20' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/12' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/10' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/11' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/16' 2024-08-18T13:43:36.554 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/14' 2024-08-18T13:43:37.186 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/18' 2024-08-18T13:43:37.186 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/13' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/190/17' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/190' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/19' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/15' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/20' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/12' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/10' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/11' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/16' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/14' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/18' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/13' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/104/17' 2024-08-18T13:43:37.187 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/104' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/19' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/15' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/20' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/12' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/10' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/11' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/16' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/14' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/18' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/13' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/164/17' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/164' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/19' 2024-08-18T13:43:37.188 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/15' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/20' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/12' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/10' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/11' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/16' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/14' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/18' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/13' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/112/17' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/112' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/19' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/15' 2024-08-18T13:43:37.189 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/20' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/12' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/10' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/11' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/16' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/14' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/18' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/13' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/100/17' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/100' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/19' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/15' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/20' 2024-08-18T13:43:37.190 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/12' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/10' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/11' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/16' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/14' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/18' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/13' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/134/17' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/134' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/19' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/15' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/20' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/12' 2024-08-18T13:43:37.191 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/10' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/11' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/16' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/14' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/18' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/13' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/115/17' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/115' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/19' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/15' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/20' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/12' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/10' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/11' 2024-08-18T13:43:37.192 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/16' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/14' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/18' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/13' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/116/17' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/116' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/19' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/15' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/20' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/12' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/10' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/11' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/16' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/14' 2024-08-18T13:43:37.193 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/18' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/13' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/195/17' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/195' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/19' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/15' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/20' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/12' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/10' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/11' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/16' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/14' 2024-08-18T13:43:37.194 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/18' 2024-08-18T13:43:37.752 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/13' 2024-08-18T13:43:37.752 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/138/17' 2024-08-18T13:43:37.752 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/138' 2024-08-18T13:43:37.752 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/19' 2024-08-18T13:43:37.752 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/15' 2024-08-18T13:43:37.752 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/20' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/12' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/10' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/11' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/16' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/14' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/18' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/13' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/135/17' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/135' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/19' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/15' 2024-08-18T13:43:37.753 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/20' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/12' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/10' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/11' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/16' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/14' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/18' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/13' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/117/17' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/117' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/19' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/15' 2024-08-18T13:43:37.754 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/20' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/12' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/10' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/11' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/16' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/14' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/18' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/13' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/143/17' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/143' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/19' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/15' 2024-08-18T13:43:37.755 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/20' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/12' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/10' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/11' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/16' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/14' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/18' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/13' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/180/17' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/180' 2024-08-18T13:43:37.756 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/19' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/15' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/20' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/12' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/10' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/11' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/16' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/14' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/18' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/13' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/126/17' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/126' 2024-08-18T13:43:37.757 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/19' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/15' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/20' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/12' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/10' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/11' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/16' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/14' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/18' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/13' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/161/17' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/161' 2024-08-18T13:43:37.758 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/19' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/15' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/20' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/12' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/10' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/11' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/16' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/14' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/18' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/13' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/176/17' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/176' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/19' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/15' 2024-08-18T13:43:37.759 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/20' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/12' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/10' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/11' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/16' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/14' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/18' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/13' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/111/17' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/111' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/19' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/15' 2024-08-18T13:43:37.760 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/20' 2024-08-18T13:43:37.761 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/12' 2024-08-18T13:43:37.761 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/10' 2024-08-18T13:43:37.761 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/11' 2024-08-18T13:43:37.761 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/16' 2024-08-18T13:43:37.761 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/14' 2024-08-18T13:43:37.761 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/18' 2024-08-18T13:43:37.761 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/13' 2024-08-18T13:43:38.466 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/191/17' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/191' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/19' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/15' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/20' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/12' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/10' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/11' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/16' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/14' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/18' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/13' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/184/17' 2024-08-18T13:43:38.467 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/184' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/19' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/15' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/20' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/12' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/10' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/11' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/16' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/14' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/18' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/13' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/192/17' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/192' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/19' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/15' 2024-08-18T13:43:38.468 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/20' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/12' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/10' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/11' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/16' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/14' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/18' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/13' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/103/17' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/103' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/19' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/15' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/20' 2024-08-18T13:43:38.469 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/12' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/10' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/11' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/16' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/14' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/18' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/13' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/128/17' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/128' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/19' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/15' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/20' 2024-08-18T13:43:38.470 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/12' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/10' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/11' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/16' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/14' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/18' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/13' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/144/17' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/144' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/19' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/15' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/20' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/12' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/10' 2024-08-18T13:43:38.471 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/11' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/16' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/14' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/18' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/13' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/146/17' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/146' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/19' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/15' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/20' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/12' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/10' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/11' 2024-08-18T13:43:38.472 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/16' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/14' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/18' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/13' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/141/17' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/141' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/19' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/15' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/20' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/12' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/10' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/11' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/16' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/14' 2024-08-18T13:43:38.473 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/18' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/13' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/182/17' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/182' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/19' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/15' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/20' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/12' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/10' 2024-08-18T13:43:38.474 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/11' 2024-08-18T13:43:38.475 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/16' 2024-08-18T13:43:38.475 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/14' 2024-08-18T13:43:38.475 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/18' 2024-08-18T13:43:38.475 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/13' 2024-08-18T13:43:38.475 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/107/17' 2024-08-18T13:43:39.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/107' 2024-08-18T13:43:39.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/19' 2024-08-18T13:43:39.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/15' 2024-08-18T13:43:39.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/20' 2024-08-18T13:43:39.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/12' 2024-08-18T13:43:39.119 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/10' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/11' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/16' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/14' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/18' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/13' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/178/17' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/178' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/19' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/15' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/20' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/12' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/10' 2024-08-18T13:43:39.120 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/11' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/16' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/14' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/18' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/13' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/156/17' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/156' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/19' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/15' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/20' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/12' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/10' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/11' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/16' 2024-08-18T13:43:39.121 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/14' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/18' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/13' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/131/17' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/131' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/19' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/15' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/20' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/12' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/10' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/11' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/16' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/14' 2024-08-18T13:43:39.122 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/18' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/13' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/123/17' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/123' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/19' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/15' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/20' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/12' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/10' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/11' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/16' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/14' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/18' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/13' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/160/17' 2024-08-18T13:43:39.123 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/160' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/19' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/15' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/20' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/12' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/10' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/11' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/16' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/14' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/18' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/13' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/165/17' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/165' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/19' 2024-08-18T13:43:39.124 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/15' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/20' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/12' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/10' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/11' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/16' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/14' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/18' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/13' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/101/17' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/101' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/19' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/15' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/20' 2024-08-18T13:43:39.125 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/12' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/10' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/11' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/16' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/14' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/18' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/13' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/148/17' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/148' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/19' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/15' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/20' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/12' 2024-08-18T13:43:39.126 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/10' 2024-08-18T13:43:39.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/11' 2024-08-18T13:43:39.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/16' 2024-08-18T13:43:39.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/14' 2024-08-18T13:43:39.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/18' 2024-08-18T13:43:39.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/13' 2024-08-18T13:43:39.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/168/17' 2024-08-18T13:43:39.127 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/168' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/19' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/15' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/20' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/12' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/10' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/11' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/16' 2024-08-18T13:43:39.762 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/14' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/18' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/13' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/179/17' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/179' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/19' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/15' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/20' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/12' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/10' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/11' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/16' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/14' 2024-08-18T13:43:39.763 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/18' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/13' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/132/17' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/132' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/19' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/15' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/20' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/12' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/10' 2024-08-18T13:43:39.764 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/11' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/16' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/14' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/18' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/13' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/199/17' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/199' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/19' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/15' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/20' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/12' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/10' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/11' 2024-08-18T13:43:39.765 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/16' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/14' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/18' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/13' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/163/17' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/163' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/19' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/15' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/20' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/12' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/10' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/11' 2024-08-18T13:43:39.766 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/16' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/14' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/18' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/13' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/153/17' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/153' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/19' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/15' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/20' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/12' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/10' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/11' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/16' 2024-08-18T13:43:39.767 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/14' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/18' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/13' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/122/17' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/122' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/19' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/15' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/20' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/12' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/10' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/11' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/16' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/14' 2024-08-18T13:43:39.768 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/18' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/13' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/162/17' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/162' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/19' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/15' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/20' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/12' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/10' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/11' 2024-08-18T13:43:39.769 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/16' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/14' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/18' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/13' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/197/17' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/197' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/19' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/15' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/20' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/12' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/10' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/11' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/16' 2024-08-18T13:43:39.770 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/14' 2024-08-18T13:43:39.771 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/18' 2024-08-18T13:43:39.771 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/13' 2024-08-18T13:43:39.771 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/113/17' 2024-08-18T13:43:39.771 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/113' 2024-08-18T13:43:39.771 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/19' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/15' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/20' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/12' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/10' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/11' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/16' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/14' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/18' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/13' 2024-08-18T13:43:40.448 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/152/17' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/152' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/19' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/15' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/20' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/12' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/10' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/11' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/16' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/14' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/18' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/13' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/121/17' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/121' 2024-08-18T13:43:40.449 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/19' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/15' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/20' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/12' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/10' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/11' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/16' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/14' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/18' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/13' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/118/17' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/118' 2024-08-18T13:43:40.450 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/19' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/15' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/20' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/12' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/10' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/11' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/16' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/14' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/18' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/13' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/196/17' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/196' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/19' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/15' 2024-08-18T13:43:40.451 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/20' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/12' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/10' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/11' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/16' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/14' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/18' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/13' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/124/17' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/124' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/19' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/15' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/20' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/12' 2024-08-18T13:43:40.452 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/10' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/11' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/16' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/14' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/18' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/13' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/188/17' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/188' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/19' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/15' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/20' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/12' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/10' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/11' 2024-08-18T13:43:40.453 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/16' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/14' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/18' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/13' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/145/17' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/145' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/19' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/15' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/20' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/12' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/10' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/11' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/16' 2024-08-18T13:43:40.454 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/14' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/18' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/13' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/130/17' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/130' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/19' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/15' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/20' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/12' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/10' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/11' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/16' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/14' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/18' 2024-08-18T13:43:40.455 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/13' 2024-08-18T13:43:40.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/193/17' 2024-08-18T13:43:40.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/193' 2024-08-18T13:43:40.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/19' 2024-08-18T13:43:40.456 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/15' 2024-08-18T13:43:41.043 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/20' 2024-08-18T13:43:41.043 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/12' 2024-08-18T13:43:41.043 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/10' 2024-08-18T13:43:41.043 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/11' 2024-08-18T13:43:41.043 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/16' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/14' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/18' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/13' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/175/17' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/175' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/19' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/15' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/20' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/12' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/10' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/11' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/16' 2024-08-18T13:43:41.044 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/14' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/18' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/13' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/181/17' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/181' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/19' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/15' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/20' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/12' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/10' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/11' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/16' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/14' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/18' 2024-08-18T13:43:41.045 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/13' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/127/17' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/127' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/19' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/15' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/20' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/12' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/10' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/11' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/16' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/14' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/18' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/13' 2024-08-18T13:43:41.046 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/108/17' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/108' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/19' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/15' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/20' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/12' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/10' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/11' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/16' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/14' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/18' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/13' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/183/17' 2024-08-18T13:43:41.047 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/183' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/19' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/15' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/20' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/12' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/10' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/11' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/16' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/14' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/18' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/13' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/177/17' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/177' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/19' 2024-08-18T13:43:41.048 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/15' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/20' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/12' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/10' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/11' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/16' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/14' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/18' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/13' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/169/17' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/169' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/19' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/15' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/20' 2024-08-18T13:43:41.049 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/12' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/10' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/11' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/16' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/14' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/18' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/13' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/154/17' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/154' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/19' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/15' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/20' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/12' 2024-08-18T13:43:41.050 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/10' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/11' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/16' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/14' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/18' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/13' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/171/17' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/171' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/19' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/15' 2024-08-18T13:43:41.051 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/20' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/12' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/10' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/11' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/16' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/14' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/18' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/13' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/139/17' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/139' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/19' 2024-08-18T13:43:41.303 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/15' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/20' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/12' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/10' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/11' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/16' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/14' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/18' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/13' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/102/17' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/102' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/19' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/15' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/20' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/12' 2024-08-18T13:43:41.304 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/10' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/11' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/16' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/14' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/18' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/13' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed './copyofsnap1/test1/129/17' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1/129' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1/test1' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:removed directory './copyofsnap1' 2024-08-18T13:43:41.305 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:43:41.305 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:43:41.306 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:43:41.341 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2024-08-18T13:43:41.341 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2024-08-18T13:43:41.362 INFO:tasks.workunit.client.0.smithi118.stderr:+ set -e 2024-08-18T13:43:41.363 INFO:tasks.workunit.client.0.smithi118.stderr:+ touch foo 2024-08-18T13:43:41.368 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod +x foo 2024-08-18T13:43:41.370 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir .snap/s 2024-08-18T13:43:41.395 INFO:tasks.workunit.client.0.smithi118.stderr:+ find .snap/s/foo -executable 2024-08-18T13:43:41.395 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep foo 2024-08-18T13:43:41.411 INFO:tasks.workunit.client.0.smithi118.stdout:.snap/s/foo 2024-08-18T13:43:41.411 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir .snap/s 2024-08-18T13:43:41.429 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm foo 2024-08-18T13:43:41.440 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-08-18T13:43:41.441 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:43:41.441 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:43:41.441 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:43:41.516 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2024-08-18T13:43:41.516 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2024-08-18T13:43:41.535 INFO:tasks.workunit.client.0.smithi118.stderr:+ set -e 2024-08-18T13:43:41.535 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir foo 2024-08-18T13:43:41.543 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo x 2024-08-18T13:43:41.543 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:42.429 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:43:42.432 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:43:44.489 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir foo/.snap/ss 2024-08-18T13:43:44.536 INFO:tasks.workunit.client.0.smithi118.stderr:+ ln foo/x foo/xx 2024-08-18T13:43:44.546 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat foo/.snap/ss/x 2024-08-18T13:43:44.559 INFO:tasks.workunit.client.0.smithi118.stdout:x 2024-08-18T13:43:44.559 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir foo/.snap/ss 2024-08-18T13:43:44.578 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo a 2024-08-18T13:43:44.582 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo b 2024-08-18T13:43:44.583 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir foo/.snap/s 2024-08-18T13:43:44.603 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat foo/.snap/s/a 2024-08-18T13:43:44.620 INFO:tasks.workunit.client.0.smithi118.stderr:+ r=a 2024-08-18T13:43:44.620 INFO:tasks.workunit.client.0.smithi118.stderr:+ [ -z a ] 2024-08-18T13:43:44.621 INFO:tasks.workunit.client.0.smithi118.stderr:+ ln foo/b foo/b2 2024-08-18T13:43:44.628 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat foo/.snap/s/b 2024-08-18T13:43:44.641 INFO:tasks.workunit.client.0.smithi118.stdout:b 2024-08-18T13:43:44.642 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo this used to hang: 2024-08-18T13:43:44.642 INFO:tasks.workunit.client.0.smithi118.stdout:this used to hang: 2024-08-18T13:43:44.645 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo more 2024-08-18T13:43:44.649 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo oh, it didn't hang! good job. 2024-08-18T13:43:44.649 INFO:tasks.workunit.client.0.smithi118.stdout:oh, it didn't hang! good job. 2024-08-18T13:43:44.649 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat foo/b 2024-08-18T13:43:44.651 INFO:tasks.workunit.client.0.smithi118.stdout:b 2024-08-18T13:43:44.651 INFO:tasks.workunit.client.0.smithi118.stdout:more 2024-08-18T13:43:44.652 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir foo/.snap/s 2024-08-18T13:43:44.670 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm -r foo 2024-08-18T13:43:44.707 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-08-18T13:43:44.708 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:43:44.708 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:43:44.708 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:43:44.740 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2024-08-18T13:43:44.741 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2024-08-18T13:43:44.759 INFO:tasks.workunit.client.0.smithi118.stderr:+ set -e 2024-08-18T13:43:44.759 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:44.763 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir .snap/bar 2024-08-18T13:43:44.784 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/foo 2024-08-18T13:43:44.799 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/foo 2024-08-18T13:43:44.816 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:44.823 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/bar/a 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout: File: .snap/bar/a 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout: Size: 0 Blocks: 1 IO Block: 4096 directory 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout:Device: 34h/52d Inode: 845524441765765 Links: 2 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1267/ ubuntu) 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout:Access: 2024-08-18 13:43:44.760917280 +0000 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout:Modify: 2024-08-18 13:43:44.760917280 +0000 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout:Change: 2024-08-18 13:43:44.760917280 +0000 2024-08-18T13:43:44.835 INFO:tasks.workunit.client.0.smithi118.stdout: Birth: - 2024-08-18T13:43:44.836 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir .snap/bar 2024-08-18T13:43:44.852 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-08-18T13:43:44.853 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:43:44.853 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:43:44.853 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:43:44.885 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2024-08-18T13:43:44.885 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2024-08-18T13:43:44.908 INFO:tasks.workunit.client.0.smithi118.stderr:+ set -e 2024-08-18T13:43:44.908 INFO:tasks.workunit.client.0.smithi118.stderr:+ seq 1 20 2024-08-18T13:43:44.909 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:44.914 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:44.919 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:44.942 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:44.957 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:44.960 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:45.181 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:45.231 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:45.231 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:45.231 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:45.232 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:45.232 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:45.232 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:45.232 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:45.232 INFO:tasks.workunit.client.0.smithi118.stdout: 35109 ? 00:00:00 ps 2024-08-18T13:43:45.241 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:45.272 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:45.282 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:45.289 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:45.293 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:45.297 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:45.314 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:45.328 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:45.330 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:45.389 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:45.436 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:45.437 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:45.437 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:45.437 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:45.437 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:45.437 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:45.437 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:45.437 INFO:tasks.workunit.client.0.smithi118.stdout: 35119 ? 00:00:00 ps 2024-08-18T13:43:45.445 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:45.473 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:45.478 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:45.487 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:45.491 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:45.495 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:45.516 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:45.528 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:45.531 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:45.597 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:45.642 INFO:tasks.workunit.client.0.smithi118.stdout: 35129 ? 00:00:00 ps 2024-08-18T13:43:45.647 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:45.682 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:45.687 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:45.692 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:45.697 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:45.701 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:45.717 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:45.733 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:45.735 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:45.846 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:45.889 INFO:tasks.workunit.client.0.smithi118.stdout: 35139 ? 00:00:00 ps 2024-08-18T13:43:45.899 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:45.921 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:45.926 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:45.933 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:45.937 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:45.941 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:45.959 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:45.972 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:45.975 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:46.064 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:46.112 INFO:tasks.workunit.client.0.smithi118.stdout: 35149 ? 00:00:00 ps 2024-08-18T13:43:46.120 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:46.150 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:46.155 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:46.163 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:46.167 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:46.171 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:46.191 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:46.211 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:46.214 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:46.348 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:46.409 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:46.410 INFO:tasks.workunit.client.0.smithi118.stdout: 35159 ? 00:00:00 ps 2024-08-18T13:43:46.415 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:46.440 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:46.446 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:46.455 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:46.460 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:46.464 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:46.481 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:46.495 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:46.498 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:46.589 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:46.654 INFO:tasks.workunit.client.0.smithi118.stdout: 35169 ? 00:00:00 ps 2024-08-18T13:43:46.658 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:46.725 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:46.733 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:46.742 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:46.743 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:46.745 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:46.755 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:46.764 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:46.765 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:47.014 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:47.063 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:47.063 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:47.063 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:47.063 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:47.064 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:47.064 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:47.064 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:47.064 INFO:tasks.workunit.client.0.smithi118.stdout: 35179 ? 00:00:00 ps 2024-08-18T13:43:47.072 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:47.105 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:47.112 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:47.125 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:47.130 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:47.132 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:47.145 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:47.156 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:47.158 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:47.230 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:47.278 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:47.278 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:47.278 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:47.278 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:47.279 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:47.279 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:47.279 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:47.279 INFO:tasks.workunit.client.0.smithi118.stdout: 35189 ? 00:00:00 ps 2024-08-18T13:43:47.286 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:47.313 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:47.319 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:47.329 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:47.333 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:47.337 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:47.354 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:47.368 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:47.371 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:47.440 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:47.491 INFO:tasks.workunit.client.0.smithi118.stdout: 35199 ? 00:00:00 ps 2024-08-18T13:43:47.497 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:47.522 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:47.530 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:47.537 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:47.541 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:47.545 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:47.566 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:47.581 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:47.584 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:47.631 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:47.681 INFO:tasks.workunit.client.0.smithi118.stdout: 35209 ? 00:00:00 ps 2024-08-18T13:43:47.687 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:47.740 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:47.743 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:47.748 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:47.752 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:47.756 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:47.780 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:47.794 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:47.797 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:48.165 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:48.213 INFO:tasks.workunit.client.0.smithi118.stdout: 35219 ? 00:00:00 ps 2024-08-18T13:43:48.220 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:48.252 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:48.259 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:48.269 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:48.274 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:48.278 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:48.299 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:48.315 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:48.318 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:48.406 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:48.455 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:48.455 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:48.456 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:48.456 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:48.456 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:48.456 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:48.456 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:48.456 INFO:tasks.workunit.client.0.smithi118.stdout: 35229 ? 00:00:00 ps 2024-08-18T13:43:48.464 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:48.495 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:48.502 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:48.511 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:48.515 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:48.519 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:48.540 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:48.556 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:48.559 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:48.632 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:48.683 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:48.683 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:48.683 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:48.683 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:48.683 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:48.683 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:48.684 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:48.684 INFO:tasks.workunit.client.0.smithi118.stdout: 35239 ? 00:00:00 ps 2024-08-18T13:43:48.687 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:48.730 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:48.736 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:48.742 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:48.748 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:48.754 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:48.770 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:48.784 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:48.786 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:49.007 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:49.059 INFO:tasks.workunit.client.0.smithi118.stdout: 35249 ? 00:00:00 ps 2024-08-18T13:43:49.066 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:49.093 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:49.097 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:49.106 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:49.110 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:49.114 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:49.129 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:49.140 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:49.150 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:49.409 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:49.458 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:49.458 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:49.458 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:49.459 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:49.459 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:49.459 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:49.459 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:49.459 INFO:tasks.workunit.client.0.smithi118.stdout: 35259 ? 00:00:00 ps 2024-08-18T13:43:49.467 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:49.501 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:49.508 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:49.517 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:49.521 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:49.524 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:49.542 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:49.556 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:49.559 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:49.634 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:49.680 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:49.681 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:49.681 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:49.681 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:49.681 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:49.681 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:49.681 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:49.681 INFO:tasks.workunit.client.0.smithi118.stdout: 35269 ? 00:00:00 ps 2024-08-18T13:43:49.686 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:49.728 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:49.732 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:49.737 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:49.741 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:49.749 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:49.762 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:49.782 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:49.788 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:50.041 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:50.089 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:50.089 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:50.089 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:50.089 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:50.089 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:50.089 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:50.089 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:50.090 INFO:tasks.workunit.client.0.smithi118.stdout: 35279 ? 00:00:00 ps 2024-08-18T13:43:50.094 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:50.126 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:50.133 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:50.146 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:50.150 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:50.154 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:50.170 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:50.179 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:50.180 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:50.325 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:50.369 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:50.369 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:50.369 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:50.370 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:50.370 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:50.370 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:50.370 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:50.370 INFO:tasks.workunit.client.0.smithi118.stdout: 35289 ? 00:00:00 ps 2024-08-18T13:43:50.376 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:50.404 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:50.409 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:50.416 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a 2024-08-18T13:43:50.421 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/one 2024-08-18T13:43:50.424 INFO:tasks.workunit.client.0.smithi118.stderr:+ cat 2024-08-18T13:43:50.442 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir a/.snap/two 2024-08-18T13:43:50.456 INFO:tasks.workunit.client.0.smithi118.stderr:+ chmod 777 a/foo 2024-08-18T13:43:50.459 INFO:tasks.workunit.client.0.smithi118.stderr:+ sync 2024-08-18T13:43:50.517 INFO:tasks.workunit.client.0.smithi118.stderr:+ ps 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: PID TTY TIME CMD 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: 1331 ? 00:00:00 systemd 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: 1333 ? 00:00:00 (sd-pam) 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: 13673 ? 00:00:01 sshd 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: 32118 ? 00:00:00 bash 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: 35089 ? 00:00:00 timeout 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: 35092 ? 00:00:00 snaptest-double 2024-08-18T13:43:50.564 INFO:tasks.workunit.client.0.smithi118.stdout: 35299 ? 00:00:00 ps 2024-08-18T13:43:50.572 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a/.snap/one a/.snap/two 2024-08-18T13:43:50.601 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm a/foo 2024-08-18T13:43:50.606 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir a 2024-08-18T13:43:50.621 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:43:50.622 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-08-18T13:43:50.622 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:43:50.622 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:43:50.695 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2024-08-18T13:43:50.695 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2024-08-18T13:43:50.739 INFO:tasks.workunit.client.0.smithi118.stderr:+ mkdir .snap/foo 2024-08-18T13:43:50.756 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo We want ENOENT, not ESTALE, here. 2024-08-18T13:43:50.756 INFO:tasks.workunit.client.0.smithi118.stdout:We want ENOENT, not ESTALE, here. 2024-08-18T13:43:50.757 INFO:tasks.workunit.client.0.smithi118.stderr:+ seq 1 100 2024-08-18T13:43:50.763 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/1 2024-08-18T13:43:50.774 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.776 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/1': No such file or directory 2024-08-18T13:43:50.778 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/2 2024-08-18T13:43:50.783 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.788 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/2': No such file or directory 2024-08-18T13:43:50.789 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/3 2024-08-18T13:43:50.789 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.797 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/3': No such file or directory 2024-08-18T13:43:50.798 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.803 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/4 2024-08-18T13:43:50.809 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/4': No such file or directory 2024-08-18T13:43:50.810 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/5 2024-08-18T13:43:50.810 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.815 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/5': No such file or directory 2024-08-18T13:43:50.816 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.817 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/6 2024-08-18T13:43:50.823 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/6': No such file or directory 2024-08-18T13:43:50.824 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.824 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/7 2024-08-18T13:43:50.827 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/7': No such file or directory 2024-08-18T13:43:50.827 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/8 2024-08-18T13:43:50.827 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.829 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/8': No such file or directory 2024-08-18T13:43:50.830 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/9 2024-08-18T13:43:50.830 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.832 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/9': No such file or directory 2024-08-18T13:43:50.832 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/10 2024-08-18T13:43:50.832 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.834 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/10': No such file or directory 2024-08-18T13:43:50.835 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/11 2024-08-18T13:43:50.835 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.837 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/11': No such file or directory 2024-08-18T13:43:50.838 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/12 2024-08-18T13:43:50.838 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.841 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/12': No such file or directory 2024-08-18T13:43:50.842 INFO:tasks.workunit.client.0.smithi118.stderr:+ + grep No such file 2024-08-18T13:43:50.842 INFO:tasks.workunit.client.0.smithi118.stderr:stat .snap/foo/13 2024-08-18T13:43:50.846 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/13': No such file or directory 2024-08-18T13:43:50.846 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/14 2024-08-18T13:43:50.846 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.851 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/14': No such file or directory 2024-08-18T13:43:50.851 INFO:tasks.workunit.client.0.smithi118.stderr:+ + grep No such file 2024-08-18T13:43:50.851 INFO:tasks.workunit.client.0.smithi118.stderr:stat .snap/foo/15 2024-08-18T13:43:50.856 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/15': No such file or directory 2024-08-18T13:43:50.857 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.857 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/16 2024-08-18T13:43:50.863 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/16': No such file or directory 2024-08-18T13:43:50.863 INFO:tasks.workunit.client.0.smithi118.stderr:+ + grep No such file 2024-08-18T13:43:50.863 INFO:tasks.workunit.client.0.smithi118.stderr:stat .snap/foo/17 2024-08-18T13:43:50.868 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/17': No such file or directory 2024-08-18T13:43:50.869 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/18 2024-08-18T13:43:50.869 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.875 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/18': No such file or directory 2024-08-18T13:43:50.876 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.876 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/19 2024-08-18T13:43:50.881 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/19': No such file or directory 2024-08-18T13:43:50.882 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/20 2024-08-18T13:43:50.882 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.887 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/20': No such file or directory 2024-08-18T13:43:50.888 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/21 2024-08-18T13:43:50.888 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.894 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/21': No such file or directory 2024-08-18T13:43:50.895 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/22 2024-08-18T13:43:50.895 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.900 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/22': No such file or directory 2024-08-18T13:43:50.900 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/23 2024-08-18T13:43:50.901 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.906 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/23': No such file or directory 2024-08-18T13:43:50.907 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/24 2024-08-18T13:43:50.907 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.913 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/24': No such file or directory 2024-08-18T13:43:50.913 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/25 2024-08-18T13:43:50.913 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.919 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/25': No such file or directory 2024-08-18T13:43:50.920 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/26 2024-08-18T13:43:50.920 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.925 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/26': No such file or directory 2024-08-18T13:43:50.925 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/27 2024-08-18T13:43:50.925 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.932 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/27': No such file or directory 2024-08-18T13:43:50.932 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/28 2024-08-18T13:43:50.933 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.939 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/28': No such file or directory 2024-08-18T13:43:50.940 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/29 2024-08-18T13:43:50.940 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.946 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/29': No such file or directory 2024-08-18T13:43:50.947 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/30 2024-08-18T13:43:50.947 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.952 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/30': No such file or directory 2024-08-18T13:43:50.953 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/31 2024-08-18T13:43:50.953 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.959 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/31': No such file or directory 2024-08-18T13:43:50.960 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/32 2024-08-18T13:43:50.960 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.966 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/32': No such file or directory 2024-08-18T13:43:50.967 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/33 2024-08-18T13:43:50.967 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.973 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/33': No such file or directory 2024-08-18T13:43:50.973 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/34 2024-08-18T13:43:50.973 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.980 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/34': No such file or directory 2024-08-18T13:43:50.981 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/35 2024-08-18T13:43:50.981 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.987 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/35': No such file or directory 2024-08-18T13:43:50.988 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/36 2024-08-18T13:43:50.988 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:50.994 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/36': No such file or directory 2024-08-18T13:43:50.994 INFO:tasks.workunit.client.0.smithi118.stderr:+ + stat .snap/foo/37 2024-08-18T13:43:50.994 INFO:tasks.workunit.client.0.smithi118.stderr:grep No such file 2024-08-18T13:43:51.001 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/37': No such file or directory 2024-08-18T13:43:51.001 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/38 2024-08-18T13:43:51.001 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.010 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/38': No such file or directory 2024-08-18T13:43:51.011 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/39 2024-08-18T13:43:51.011 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.015 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/39': No such file or directory 2024-08-18T13:43:51.079 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.079 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/40 2024-08-18T13:43:51.080 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/40': No such file or directory 2024-08-18T13:43:51.080 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/41 2024-08-18T13:43:51.080 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.081 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/41': No such file or directory 2024-08-18T13:43:51.081 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/42 2024-08-18T13:43:51.081 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.081 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/42': No such file or directory 2024-08-18T13:43:51.082 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat+ .snap/foo/43 2024-08-18T13:43:51.082 INFO:tasks.workunit.client.0.smithi118.stderr:grep No such file 2024-08-18T13:43:51.082 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/43': No such file or directory 2024-08-18T13:43:51.083 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/44 2024-08-18T13:43:51.083 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.083 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/44': No such file or directory 2024-08-18T13:43:51.083 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/45 2024-08-18T13:43:51.084 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.084 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/45': No such file or directory 2024-08-18T13:43:51.084 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/46 2024-08-18T13:43:51.084 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.085 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/46': No such file or directory 2024-08-18T13:43:51.085 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/47 2024-08-18T13:43:51.085 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.086 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/47': No such file or directory 2024-08-18T13:43:51.086 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/48 2024-08-18T13:43:51.086 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.086 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/48': No such file or directory 2024-08-18T13:43:51.087 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/49 2024-08-18T13:43:51.087 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.089 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/49': No such file or directory 2024-08-18T13:43:51.090 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/50 2024-08-18T13:43:51.090 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.096 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/50': No such file or directory 2024-08-18T13:43:51.097 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/51 2024-08-18T13:43:51.097 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.103 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/51': No such file or directory 2024-08-18T13:43:51.104 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/52 2024-08-18T13:43:51.104 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.110 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/52': No such file or directory 2024-08-18T13:43:51.111 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/53 2024-08-18T13:43:51.111 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.119 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/53': No such file or directory 2024-08-18T13:43:51.120 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/54 2024-08-18T13:43:51.120 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.128 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/54': No such file or directory 2024-08-18T13:43:51.129 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/55 2024-08-18T13:43:51.129 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.132 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/55': No such file or directory 2024-08-18T13:43:51.133 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/56 2024-08-18T13:43:51.133 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.136 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/56': No such file or directory 2024-08-18T13:43:51.137 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/57 2024-08-18T13:43:51.137 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.140 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/57': No such file or directory 2024-08-18T13:43:51.141 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/58 2024-08-18T13:43:51.141 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.145 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/58': No such file or directory 2024-08-18T13:43:51.145 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/59 2024-08-18T13:43:51.145 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.150 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/59': No such file or directory 2024-08-18T13:43:51.150 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/60 2024-08-18T13:43:51.150 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.155 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/60': No such file or directory 2024-08-18T13:43:51.156 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/61 2024-08-18T13:43:51.156 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.161 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/61': No such file or directory 2024-08-18T13:43:51.161 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/62 2024-08-18T13:43:51.161 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.167 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/62': No such file or directory 2024-08-18T13:43:51.168 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/63 2024-08-18T13:43:51.168 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.173 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/63': No such file or directory 2024-08-18T13:43:51.173 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/64 2024-08-18T13:43:51.173 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.179 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/64': No such file or directory 2024-08-18T13:43:51.180 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/65 2024-08-18T13:43:51.180 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.187 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/65': No such file or directory 2024-08-18T13:43:51.187 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/66 2024-08-18T13:43:51.187 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.194 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/66': No such file or directory 2024-08-18T13:43:51.195 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/67 2024-08-18T13:43:51.195 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.201 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/67': No such file or directory 2024-08-18T13:43:51.201 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/68 2024-08-18T13:43:51.201 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.208 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/68': No such file or directory 2024-08-18T13:43:51.208 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/69 2024-08-18T13:43:51.209 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.215 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/69': No such file or directory 2024-08-18T13:43:51.215 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/70 2024-08-18T13:43:51.215 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.221 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/70': No such file or directory 2024-08-18T13:43:51.222 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/71 2024-08-18T13:43:51.222 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.228 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/71': No such file or directory 2024-08-18T13:43:51.228 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/72 2024-08-18T13:43:51.228 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.234 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/72': No such file or directory 2024-08-18T13:43:51.235 INFO:tasks.workunit.client.0.smithi118.stderr:+ + stat .snap/foo/73 2024-08-18T13:43:51.235 INFO:tasks.workunit.client.0.smithi118.stderr:grep No such file 2024-08-18T13:43:51.241 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/73': No such file or directory 2024-08-18T13:43:51.242 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/74 2024-08-18T13:43:51.242 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.249 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/74': No such file or directory 2024-08-18T13:43:51.249 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/75 2024-08-18T13:43:51.249 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.255 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/75': No such file or directory 2024-08-18T13:43:51.256 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/76 2024-08-18T13:43:51.256 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.261 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/76': No such file or directory 2024-08-18T13:43:51.262 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/77 2024-08-18T13:43:51.262 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.268 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/77': No such file or directory 2024-08-18T13:43:51.269 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/78 2024-08-18T13:43:51.269 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.275 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/78': No such file or directory 2024-08-18T13:43:51.276 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/79 2024-08-18T13:43:51.276 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.283 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/79': No such file or directory 2024-08-18T13:43:51.283 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/80 2024-08-18T13:43:51.284 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.290 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/80': No such file or directory 2024-08-18T13:43:51.291 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/81 2024-08-18T13:43:51.291 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.297 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/81': No such file or directory 2024-08-18T13:43:51.297 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/82 2024-08-18T13:43:51.298 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.304 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/82': No such file or directory 2024-08-18T13:43:51.305 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/83 2024-08-18T13:43:51.305 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.311 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/83': No such file or directory 2024-08-18T13:43:51.312 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/84 2024-08-18T13:43:51.312 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.318 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/84': No such file or directory 2024-08-18T13:43:51.319 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/85 2024-08-18T13:43:51.319 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.325 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/85': No such file or directory 2024-08-18T13:43:51.326 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/86 2024-08-18T13:43:51.326 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.332 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/86': No such file or directory 2024-08-18T13:43:51.333 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/87 2024-08-18T13:43:51.333 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.338 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/87': No such file or directory 2024-08-18T13:43:51.339 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/88 2024-08-18T13:43:51.339 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.345 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/88': No such file or directory 2024-08-18T13:43:51.346 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/89 2024-08-18T13:43:51.346 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.352 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/89': No such file or directory 2024-08-18T13:43:51.353 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/90 2024-08-18T13:43:51.353 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.360 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/90': No such file or directory 2024-08-18T13:43:51.360 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/91 2024-08-18T13:43:51.360 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.366 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/91': No such file or directory 2024-08-18T13:43:51.367 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/92 2024-08-18T13:43:51.367 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.373 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/92': No such file or directory 2024-08-18T13:43:51.374 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/93 2024-08-18T13:43:51.374 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.380 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/93': No such file or directory 2024-08-18T13:43:51.381 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/94 2024-08-18T13:43:51.381 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.387 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/94': No such file or directory 2024-08-18T13:43:51.387 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/95 2024-08-18T13:43:51.387 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.393 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/95': No such file or directory 2024-08-18T13:43:51.394 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/96 2024-08-18T13:43:51.394 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.400 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/96': No such file or directory 2024-08-18T13:43:51.401 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/97 2024-08-18T13:43:51.401 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.407 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/97': No such file or directory 2024-08-18T13:43:51.408 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/98 2024-08-18T13:43:51.408 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.418 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/98': No such file or directory 2024-08-18T13:43:51.419 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/99 2024-08-18T13:43:51.419 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.421 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/99': No such file or directory 2024-08-18T13:43:51.422 INFO:tasks.workunit.client.0.smithi118.stderr:+ stat .snap/foo/100 2024-08-18T13:43:51.422 INFO:tasks.workunit.client.0.smithi118.stderr:+ grep No such file 2024-08-18T13:43:51.425 INFO:tasks.workunit.client.0.smithi118.stdout:stat: cannot stat '.snap/foo/100': No such file or directory 2024-08-18T13:43:51.425 INFO:tasks.workunit.client.0.smithi118.stderr:+ rmdir .snap/foo 2024-08-18T13:43:51.437 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-08-18T13:43:51.438 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-08-18T13:43:51.438 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T13:43:51.438 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T13:43:51.460 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2024-08-18T13:43:51.460 DEBUG:teuthology.orchestra.run.smithi118:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2024-08-18T13:43:51.479 INFO:tasks.workunit.client.0.smithi118.stderr:+ set -e 2024-08-18T13:43:51.479 INFO:tasks.workunit.client.0.smithi118.stderr:+ sudo git config --global http.sslVerify false 2024-08-18T13:43:51.499 INFO:tasks.workunit.client.0.smithi118.stderr:+ sudo git config --global http.postBuffer 1048576000 2024-08-18T13:43:51.517 INFO:tasks.workunit.client.0.smithi118.stderr:+ retried=false 2024-08-18T13:43:51.517 INFO:tasks.workunit.client.0.smithi118.stderr:+ trap -- retry EXIT 2024-08-18T13:43:51.517 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm -rf ceph 2024-08-18T13:43:51.519 INFO:tasks.workunit.client.0.smithi118.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2024-08-18T13:43:51.528 INFO:tasks.workunit.client.0.smithi118.stderr:Cloning into 'ceph'... 2024-08-18T13:44:01.769 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:01.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.873+0000 7f79e2c21700 1 Processor -- start 2024-08-18T13:44:01.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.873+0000 7f79e2c21700 1 -- start start 2024-08-18T13:44:01.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e2c21700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06b810 0x7f79dc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:01.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e2c21700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79dc06c120 0x7f79dc067a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:01.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e2c21700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f79dc0c8e90 0x7f79dc067fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:01.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e2c21700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc0bfd10 con 0x7f79dc0c8e90 2024-08-18T13:44:01.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e2c21700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc152a50 con 0x7f79dc06c120 2024-08-18T13:44:01.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e2c21700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc152520 con 0x7f79dc06b810 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f79dc0c8e90 0x7f79dc067fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f79dc0c8e90 0x7f79dc067fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:56598/0 (socket says 172.21.15.118:56598) 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 -- 172.21.15.118:0/2914418724 learned_addr learned my addr 172.21.15.118:0/2914418724 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e09bd700 1 --2- 172.21.15.118:0/2914418724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06b810 0x7f79dc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79d3fff700 1 --2- 172.21.15.118:0/2914418724 >> v2:172.21.15.178:3300/0 conn(0x7f79dc06c120 0x7f79dc067a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 -- 172.21.15.118:0/2914418724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06b810 msgr2=0x7f79dc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 --2- 172.21.15.118:0/2914418724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06b810 0x7f79dc06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 -- 172.21.15.118:0/2914418724 >> v2:172.21.15.178:3300/0 conn(0x7f79dc06c120 msgr2=0x7f79dc067a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 --2- 172.21.15.118:0/2914418724 >> v2:172.21.15.178:3300/0 conn(0x7f79dc06c120 0x7f79dc067a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 -- 172.21.15.118:0/2914418724 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0c23a0 con 0x7f79dc0c8e90 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e09bd700 1 --2- 172.21.15.118:0/2914418724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06b810 0x7f79dc06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:01.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79d3fff700 1 --2- 172.21.15.118:0/2914418724 >> v2:172.21.15.178:3300/0 conn(0x7f79dc06c120 0x7f79dc067a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:01.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79e11be700 1 --2- 172.21.15.118:0/2914418724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0c8e90 0x7f79dc067fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79dc06fa30 tx=0x7f79d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cfe38ca38b02f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:01.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79d37fe700 1 -- 172.21.15.118:0/2914418724 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400e070 con 0x7f79dc0c8e90 2024-08-18T13:44:01.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79d37fe700 1 -- 172.21.15.118:0/2914418724 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400bae0 con 0x7f79dc0c8e90 2024-08-18T13:44:01.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.877+0000 7f79d37fe700 1 -- 172.21.15.118:0/2914418724 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40092e0 con 0x7f79dc0c8e90 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- 172.21.15.118:0/2914418724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0c8e90 msgr2=0x7f79dc067fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 --2- 172.21.15.118:0/2914418724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0c8e90 0x7f79dc067fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79dc06fa30 tx=0x7f79d4009840 comp rx=0 tx=0).stop 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- 172.21.15.118:0/2914418724 shutdown_connections 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 --2- 172.21.15.118:0/2914418724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc0c8e90 0x7f79dc067fd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 --2- 172.21.15.118:0/2914418724 >> v2:172.21.15.178:3300/0 conn(0x7f79dc06c120 0x7f79dc067a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 --2- 172.21.15.118:0/2914418724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06b810 0x7f79dc06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- 172.21.15.118:0/2914418724 >> 172.21.15.118:0/2914418724 conn(0x7f79dc0c8380 msgr2=0x7f79dc0c8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:01.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- 172.21.15.118:0/2914418724 shutdown_connections 2024-08-18T13:44:01.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- 172.21.15.118:0/2914418724 wait complete. 2024-08-18T13:44:01.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 Processor -- start 2024-08-18T13:44:01.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- start start 2024-08-18T13:44:01.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc06b810 0x7f79dc155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:01.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06c120 0x7f79dc1558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:01.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79dc0c8e90 0x7f79dc150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:01.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc153b60 con 0x7f79dc06b810 2024-08-18T13:44:01.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc153620 con 0x7f79dc0c8e90 2024-08-18T13:44:01.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e2c21700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc1538c0 con 0x7f79dc06c120 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e09bd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc06b810 0x7f79dc155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e09bd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc06b810 0x7f79dc155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:56602/0 (socket says 172.21.15.118:56602) 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e09bd700 1 -- 172.21.15.118:0/3013358495 learned_addr learned my addr 172.21.15.118:0/3013358495 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79d3fff700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06c120 0x7f79dc1558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.881+0000 7f79e11be700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79dc0c8e90 0x7f79dc150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e09bd700 1 -- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06c120 msgr2=0x7f79dc1558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e09bd700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06c120 0x7f79dc1558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e09bd700 1 -- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79dc0c8e90 msgr2=0x7f79dc150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e09bd700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79dc0c8e90 0x7f79dc150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e09bd700 1 -- 172.21.15.118:0/3013358495 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4009ae0 con 0x7f79dc06b810 2024-08-18T13:44:01.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e09bd700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc06b810 0x7f79dc155370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e8d0 tx=0x7f79d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:01.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79d0ff9700 1 -- 172.21.15.118:0/3013358495 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8013410 con 0x7f79dc06b810 2024-08-18T13:44:01.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79d0ff9700 1 -- 172.21.15.118:0/3013358495 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d8013570 con 0x7f79dc06b810 2024-08-18T13:44:01.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc14f570 con 0x7f79dc06b810 2024-08-18T13:44:01.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79d0ff9700 1 -- 172.21.15.118:0/3013358495 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8013910 con 0x7f79dc06b810 2024-08-18T13:44:01.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc150170 con 0x7f79dc06b810 2024-08-18T13:44:01.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc14ffd0 con 0x7f79dc06b810 2024-08-18T13:44:01.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79d0ff9700 1 -- 172.21.15.118:0/3013358495 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f79d8013cc0 con 0x7f79dc06b810 2024-08-18T13:44:01.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.885+0000 7f79d0ff9700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f79c8060860 0x7f79c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:01.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.889+0000 7f79d3fff700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f79c8060860 0x7f79c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:01.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.889+0000 7f79d0ff9700 1 -- 172.21.15.118:0/3013358495 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f79d807cd50 con 0x7f79dc06b810 2024-08-18T13:44:01.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.889+0000 7f79d3fff700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f79c8060860 0x7f79c8062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f79cc006fd0 tx=0x7f79cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:01.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:01.897+0000 7f79d0ff9700 1 -- 172.21.15.118:0/3013358495 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f79dc14ffd0 con 0x7f79dc06b810 2024-08-18T13:44:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.097+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79dc14ffd0 con 0x7f79dc06b810 2024-08-18T13:44:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.097+0000 7f79d0ff9700 1 -- 172.21.15.118:0/3013358495 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f79dc14ffd0 con 0x7f79dc06b810 2024-08-18T13:44:02.104 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:02.104 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:02.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f79c8060860 msgr2=0x7f79c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f79c8060860 0x7f79c8062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f79cc006fd0 tx=0x7f79cc008040 comp rx=0 tx=0).stop 2024-08-18T13:44:02.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc06b810 msgr2=0x7f79dc155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc06b810 0x7f79dc155370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e8d0 tx=0x7f79d8008040 comp rx=0 tx=0).stop 2024-08-18T13:44:02.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 shutdown_connections 2024-08-18T13:44:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f79c8060860 0x7f79c8062d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79dc0c8e90 0x7f79dc150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79dc06c120 0x7f79dc1558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 --2- 172.21.15.118:0/3013358495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f79dc06b810 0x7f79dc155370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 >> 172.21.15.118:0/3013358495 conn(0x7f79dc0c8380 msgr2=0x7f79dc0c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 shutdown_connections 2024-08-18T13:44:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.101+0000 7f79e2c21700 1 -- 172.21.15.118:0/3013358495 wait complete. 2024-08-18T13:44:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:44:02.117 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2024-08-18T13:44:02.118 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-08-18T13:44:02.118 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:44:02.140 INFO:tasks.ceph.mds.f:Stopped 2024-08-18T13:44:02.140 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:02.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.241+0000 7f0040fd6700 1 Processor -- start 2024-08-18T13:44:02.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.241+0000 7f0040fd6700 1 -- start start 2024-08-18T13:44:02.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.241+0000 7f0040fd6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 0x7f003c0ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.241+0000 7f0040fd6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f003c14f7e0 0x7f003c0abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.241+0000 7f0040fd6700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c0c1c80 con 0x7f003c158830 2024-08-18T13:44:02.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.241+0000 7f0040fd6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003c0c18e0 con 0x7f003c14f7e0 2024-08-18T13:44:02.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.241+0000 7f0040fd6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0c09f0 con 0x7f003c14f410 2024-08-18T13:44:02.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003bfff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f003c158830 0x7f003c0c7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:46978/0 (socket says 172.21.15.118:46978) 2024-08-18T13:44:02.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003bfff700 1 -- 172.21.15.118:0/1917940990 learned_addr learned my addr 172.21.15.118:0/1917940990 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:02.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 --2- 172.21.15.118:0/1917940990 >> v2:172.21.15.178:3300/0 conn(0x7f003c14f7e0 0x7f003c0abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003b7fe700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 0x7f003c0ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822305635 0 0) 0x7f003c0c1c80 con 0x7f003c158830 2024-08-18T13:44:02.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0028003570 con 0x7f003c158830 2024-08-18T13:44:02.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 861337871 0 0) 0x7f0028003570 con 0x7f003c158830 2024-08-18T13:44:02.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 msgr2=0x7f003c0ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:02.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 0x7f003c0ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 >> v2:172.21.15.178:3300/0 conn(0x7f003c14f7e0 msgr2=0x7f003c0abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:02.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 --2- 172.21.15.118:0/1917940990 >> v2:172.21.15.178:3300/0 conn(0x7f003c14f7e0 0x7f003c0abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0c0cd0 con 0x7f003c158830 2024-08-18T13:44:02.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f002c001470 con 0x7f003c158830 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 >> v1:172.21.15.118:6789/0 conn(0x7f003c158830 legacy=0x7f003c0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 0x7f0028003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f0028004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0028004d50 0x7f00280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0c18e0 con 0x7f0028003fe0 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0c09f0 con 0x7f0028004d50 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002800ba90 con 0x7f003c14f410 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f0028004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003bfff700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0028004d50 0x7f00280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003b7fe700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 0x7f0028003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 -- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 msgr2=0x7f0028003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 0x7f0028003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 -- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0028004d50 msgr2=0x7f00280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0028004d50 0x7f00280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 -- 172.21.15.118:0/1917940990 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028003570 con 0x7f0028003fe0 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003bfff700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0028004d50 0x7f00280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003b7fe700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c14f410 0x7f0028003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003affd700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f0028004810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0024009c00 tx=0x7f00240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb7816eca49504c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400e6b0 con 0x7f0028003fe0 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002400e810 con 0x7f0028003fe0 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003a7fc700 1 -- 172.21.15.118:0/1917940990 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400ebd0 con 0x7f0028003fe0 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f0040fd6700 1 -- 172.21.15.118:0/1917940990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 msgr2=0x7f0028004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f0040fd6700 1 --2- 172.21.15.118:0/1917940990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f0028004810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0024009c00 tx=0x7f00240091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f003bfff700 1 -- 172.21.15.118:0/1917940990 reap_dead start 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f0040fd6700 1 -- 172.21.15.118:0/1917940990 shutdown_connections 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f0040fd6700 1 -- 172.21.15.118:0/1917940990 >> 172.21.15.118:0/1917940990 conn(0x7f003c0c8620 msgr2=0x7f003c0c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:02.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f0040fd6700 1 -- 172.21.15.118:0/1917940990 shutdown_connections 2024-08-18T13:44:02.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.245+0000 7f0040fd6700 1 -- 172.21.15.118:0/1917940990 wait complete. 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 Processor -- start 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 -- start start 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f003c1650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f003c1533e0 0x7f003c157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c1577d0 0x7f003c14c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0c3600 con 0x7f0028003fe0 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c168ec0 con 0x7f003c1533e0 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1685d0 con 0x7f003c1577d0 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f003c1650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f003c1650a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:56606/0 (socket says 172.21.15.118:56606) 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 -- 172.21.15.118:0/405445692 learned_addr learned my addr 172.21.15.118:0/405445692 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003affd700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f003c1533e0 0x7f003c157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003bfff700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c1577d0 0x7f003c14c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 -- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c1577d0 msgr2=0x7f003c14c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c1577d0 0x7f003c14c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 -- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f003c1533e0 msgr2=0x7f003c157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f003c1533e0 0x7f003c157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 -- 172.21.15.118:0/405445692 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0c23c0 con 0x7f0028003fe0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003affd700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f003c1533e0 0x7f003c157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f003b7fe700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f003c1650a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f003400f4d0 tx=0x7f003400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f00337fe700 1 -- 172.21.15.118:0/405445692 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003401a070 con 0x7f0028003fe0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f00337fe700 1 -- 172.21.15.118:0/405445692 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0034016420 con 0x7f0028003fe0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c168870 con 0x7f0028003fe0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f00337fe700 1 -- 172.21.15.118:0/405445692 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00340167a0 con 0x7f0028003fe0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.249+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c0c2e90 con 0x7f0028003fe0 2024-08-18T13:44:02.260 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.253+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c168cf0 con 0x7f0028003fe0 2024-08-18T13:44:02.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.253+0000 7f00337fe700 1 -- 172.21.15.118:0/405445692 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0034003020 con 0x7f0028003fe0 2024-08-18T13:44:02.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.253+0000 7f00337fe700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f001c060860 0x7f001c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:02.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.253+0000 7f00337fe700 1 -- 172.21.15.118:0/405445692 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f003407e8b0 con 0x7f0028003fe0 2024-08-18T13:44:02.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.253+0000 7f003affd700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f001c060860 0x7f001c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:02.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.253+0000 7f003affd700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f001c060860 0x7f001c062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00240091e0 tx=0x7f002401a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:02.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.261+0000 7f00337fe700 1 -- 172.21.15.118:0/405445692 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f003404e900 con 0x7f0028003fe0 2024-08-18T13:44:02.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f003c0713a0 con 0x7f0028003fe0 2024-08-18T13:44:02.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f00337fe700 1 -- 172.21.15.118:0/405445692 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f0034051f20 con 0x7f0028003fe0 2024-08-18T13:44:02.478 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:02.478 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:02.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f001c060860 msgr2=0x7f001c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f001c060860 0x7f001c062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00240091e0 tx=0x7f002401a000 comp rx=0 tx=0).stop 2024-08-18T13:44:02.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 msgr2=0x7f003c1650a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:02.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f003c1650a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f003400f4d0 tx=0x7f003400b040 comp rx=0 tx=0).stop 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 shutdown_connections 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f001c060860 0x7f001c062d10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f003c1577d0 0x7f003c14c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f003c1533e0 0x7f003c157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 --2- 172.21.15.118:0/405445692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0028003fe0 0x7f003c1650a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 >> 172.21.15.118:0/405445692 conn(0x7f003c0c8620 msgr2=0x7f003c14e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 shutdown_connections 2024-08-18T13:44:02.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:02.473+0000 7f0040fd6700 1 -- 172.21.15.118:0/405445692 wait complete. 2024-08-18T13:44:02.482 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:44:02.491 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:44:04.492 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:04.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.601+0000 7f144b59e700 1 Processor -- start 2024-08-18T13:44:04.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.601+0000 7f144b59e700 1 -- start start 2024-08-18T13:44:04.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f144c0c3a50 0x7f144c14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:04.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f144c0c3e70 0x7f144c14d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:04.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144c06cc90 con 0x7f144c0c3680 2024-08-18T13:44:04.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c067ae0 con 0x7f144c0c3e70 2024-08-18T13:44:04.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c14d780 con 0x7f144c0c3a50 2024-08-18T13:44:04.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f144c0c3e70 0x7f144c14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:04.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f144c0c3e70 0x7f144c14d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58202/0 (socket says 172.21.15.118:58202) 2024-08-18T13:44:04.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 -- 172.21.15.118:0/3837851478 learned_addr learned my addr 172.21.15.118:0/3837851478 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f1449d9b700 1 --2- 172.21.15.118:0/3837851478 >> v2:172.21.15.178:3300/0 conn(0x7f144c0c3a50 0x7f144c14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 -- 172.21.15.118:0/3837851478 >> v2:172.21.15.178:3300/0 conn(0x7f144c0c3a50 msgr2=0x7f144c14cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 --2- 172.21.15.118:0/3837851478 >> v2:172.21.15.178:3300/0 conn(0x7f144c0c3a50 0x7f144c14cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 -- 172.21.15.118:0/3837851478 >> v1:172.21.15.178:6789/0 conn(0x7f144c0c3680 legacy=0x7f144c0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 -- 172.21.15.118:0/3837851478 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c0c83a0 con 0x7f144c0c3e70 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144ad9d700 1 --2- 172.21.15.118:0/3837851478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c14d240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f14440099b0 tx=0x7f1444009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc05b40684c8cf8e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144959a700 1 -- 172.21.15.118:0/3837851478 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444009ea0 con 0x7f144c0c3e70 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144959a700 1 -- 172.21.15.118:0/3837851478 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144400b820 con 0x7f144c0c3e70 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144959a700 1 -- 172.21.15.118:0/3837851478 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144400bbc0 con 0x7f144c0c3e70 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- 172.21.15.118:0/3837851478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 msgr2=0x7f144c14d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 --2- 172.21.15.118:0/3837851478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c14d240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f14440099b0 tx=0x7f1444009250 comp rx=0 tx=0).stop 2024-08-18T13:44:04.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- 172.21.15.118:0/3837851478 shutdown_connections 2024-08-18T13:44:04.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 --2- 172.21.15.118:0/3837851478 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c14d240 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 --2- 172.21.15.118:0/3837851478 >> v2:172.21.15.178:3300/0 conn(0x7f144c0c3a50 0x7f144c14cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- 172.21.15.118:0/3837851478 >> 172.21.15.118:0/3837851478 conn(0x7f144c06c020 msgr2=0x7f144c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:04.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- 172.21.15.118:0/3837851478 shutdown_connections 2024-08-18T13:44:04.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.605+0000 7f144b59e700 1 -- 172.21.15.118:0/3837851478 wait complete. 2024-08-18T13:44:04.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 Processor -- start 2024-08-18T13:44:04.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 -- start start 2024-08-18T13:44:04.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f144c0c3680 0x7f144c156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:04.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f144c0c3a50 0x7f144c14cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:04.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:04.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c166cb0 con 0x7f144c0c3e70 2024-08-18T13:44:04.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c166f90 con 0x7f144c0c3680 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c1660c0 con 0x7f144c0c3a50 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58212/0 (socket says 172.21.15.118:58212) 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 -- 172.21.15.118:0/4278646349 learned_addr learned my addr 172.21.15.118:0/4278646349 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144a59c700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f144c0c3680 0x7f144c156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f1449d9b700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f144c0c3a50 0x7f144c14cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 -- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f144c0c3a50 msgr2=0x7f144c14cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f144c0c3a50 0x7f144c14cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 -- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f144c0c3680 msgr2=0x7f144c156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f144c0c3680 0x7f144c156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 -- 172.21.15.118:0/4278646349 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440094f0 con 0x7f144c0c3e70 2024-08-18T13:44:04.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144ad9d700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c149230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f14440099b0 tx=0x7f1444017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:04.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f14427fc700 1 -- 172.21.15.118:0/4278646349 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444011420 con 0x7f144c0c3e70 2024-08-18T13:44:04.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f14427fc700 1 -- 172.21.15.118:0/4278646349 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14440115a0 con 0x7f144c0c3e70 2024-08-18T13:44:04.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c166380 con 0x7f144c0c3e70 2024-08-18T13:44:04.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f14427fc700 1 -- 172.21.15.118:0/4278646349 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1444011960 con 0x7f144c0c3e70 2024-08-18T13:44:04.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c166940 con 0x7f144c0c3e70 2024-08-18T13:44:04.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.609+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c0beaf0 con 0x7f144c0c3e70 2024-08-18T13:44:04.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.613+0000 7f14427fc700 1 -- 172.21.15.118:0/4278646349 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1444011d50 con 0x7f144c0c3e70 2024-08-18T13:44:04.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.613+0000 7f14427fc700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f142c060860 0x7f142c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:04.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.613+0000 7f14427fc700 1 -- 172.21.15.118:0/4278646349 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f144407ed50 con 0x7f144c0c3e70 2024-08-18T13:44:04.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.613+0000 7f144a59c700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f142c060860 0x7f142c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:04.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.613+0000 7f144a59c700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f142c060860 0x7f142c062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14340039e0 tx=0x7f1434007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:04.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.617+0000 7f14427fc700 1 -- 172.21.15.118:0/4278646349 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f144404eda0 con 0x7f144c0c3e70 2024-08-18T13:44:04.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.817+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f144c0beaf0 con 0x7f144c0c3e70 2024-08-18T13:44:04.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.817+0000 7f14427fc700 1 -- 172.21.15.118:0/4278646349 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f144c0beaf0 con 0x7f144c0c3e70 2024-08-18T13:44:04.825 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:04.825 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f142c060860 msgr2=0x7f142c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f142c060860 0x7f142c062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14340039e0 tx=0x7f1434007ec0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 msgr2=0x7f144c149230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c149230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f14440099b0 tx=0x7f1444017000 comp rx=0 tx=0).stop 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 shutdown_connections 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f142c060860 0x7f142c062d10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f144c0c3e70 0x7f144c149230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f144c0c3a50 0x7f144c14cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 --2- 172.21.15.118:0/4278646349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f144c0c3680 0x7f144c156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:04.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 >> 172.21.15.118:0/4278646349 conn(0x7f144c06c020 msgr2=0x7f144c1519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:04.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 shutdown_connections 2024-08-18T13:44:04.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:04.821+0000 7f144b59e700 1 -- 172.21.15.118:0/4278646349 wait complete. 2024-08-18T13:44:04.828 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:44:04.838 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:44:06.839 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:06.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0f59e700 1 Processor -- start 2024-08-18T13:44:06.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0f59e700 1 -- start start 2024-08-18T13:44:06.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3a50 0x7f1e1014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:06.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1006cc90 con 0x7f1e100c3680 2024-08-18T13:44:06.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0f59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100c83a0 con 0x7f1e100c3e40 2024-08-18T13:44:06.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10067ae0 con 0x7f1e100c3a50 2024-08-18T13:44:06.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0e59c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f1e100c3680 0x7f1e100673d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37116/0 (socket says 172.21.15.118:37116) 2024-08-18T13:44:06.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0e59c700 1 -- 172.21.15.118:0/3284896681 learned_addr learned my addr 172.21.15.118:0/3284896681 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:06.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0dd9b700 1 --2- 172.21.15.118:0/3284896681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3a50 0x7f1e1014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0d59a700 1 -- 172.21.15.118:0/3284896681 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836980140 0 0) 0x7f1e1006cc90 con 0x7f1e100c3680 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.949+0000 7f1e0d59a700 1 -- 172.21.15.118:0/3284896681 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df0003570 con 0x7f1e100c3680 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0d59a700 1 -- 172.21.15.118:0/3284896681 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245187433 0 0) 0x7f1e100c83a0 con 0x7f1e100c3e40 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0dd9b700 1 -- 172.21.15.118:0/3284896681 >> v1:172.21.15.178:6789/0 conn(0x7f1e100c3e40 legacy=0x7f1e1014d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0dd9b700 1 -- 172.21.15.118:0/3284896681 >> v1:172.21.15.118:6789/0 conn(0x7f1e100c3680 legacy=0x7f1e100673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0dd9b700 1 -- 172.21.15.118:0/3284896681 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10153090 con 0x7f1e100c3a50 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0dd9b700 1 --2- 172.21.15.118:0/3284896681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3a50 0x7f1e1014cd00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009910 tx=0x7f1df80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc479c5b11cfa60 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0d59a700 1 -- 172.21.15.118:0/3284896681 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df800b680 con 0x7f1e100c3a50 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0d59a700 1 -- 172.21.15.118:0/3284896681 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df800b7e0 con 0x7f1e100c3a50 2024-08-18T13:44:06.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0d59a700 1 -- 172.21.15.118:0/3284896681 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df800bb60 con 0x7f1e100c3a50 2024-08-18T13:44:06.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 -- 172.21.15.118:0/3284896681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3a50 msgr2=0x7f1e1014cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:06.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/3284896681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3a50 0x7f1e1014cd00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009910 tx=0x7f1df80091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:06.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 -- 172.21.15.118:0/3284896681 shutdown_connections 2024-08-18T13:44:06.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/3284896681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3a50 0x7f1e1014cd00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:06.960 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 -- 172.21.15.118:0/3284896681 >> 172.21.15.118:0/3284896681 conn(0x7f1e1006c020 msgr2=0x7f1e100c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:06.960 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 -- 172.21.15.118:0/3284896681 shutdown_connections 2024-08-18T13:44:06.960 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 -- 172.21.15.118:0/3284896681 wait complete. 2024-08-18T13:44:06.960 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 Processor -- start 2024-08-18T13:44:06.960 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.953+0000 7f1e0f59e700 1 -- start start 2024-08-18T13:44:06.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e100c3680 0x7f1e100bfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:06.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e100c3a50 0x7f1e10158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:06.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3e40 0x7f1e10168360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:06.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100c6e20 con 0x7f1e100c3680 2024-08-18T13:44:06.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10164de0 con 0x7f1e100c3a50 2024-08-18T13:44:06.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100c5850 con 0x7f1e100c3e40 2024-08-18T13:44:06.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e100c3680 0x7f1e100bfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:06.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0ed9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3e40 0x7f1e10168360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:06.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e100c3680 0x7f1e100bfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58222/0 (socket says 172.21.15.118:58222) 2024-08-18T13:44:06.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 -- 172.21.15.118:0/4154769105 learned_addr learned my addr 172.21.15.118:0/4154769105 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:06.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0dd9b700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e100c3a50 0x7f1e10158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:06.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 -- 172.21.15.118:0/4154769105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3e40 msgr2=0x7f1e10168360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:06.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3e40 0x7f1e10168360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:06.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 -- 172.21.15.118:0/4154769105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e100c3a50 msgr2=0x7f1e10158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e100c3a50 0x7f1e10158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 -- 172.21.15.118:0/4154769105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009420 con 0x7f1e100c3680 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0e59c700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e100c3680 0x7f1e100bfa80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e000011e0 tx=0x7f1e00007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e067fc700 1 -- 172.21.15.118:0/4154769105 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00005b30 con 0x7f1e100c3680 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e067fc700 1 -- 172.21.15.118:0/4154769105 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00005c90 con 0x7f1e100c3680 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e10164ff0 con 0x7f1e100c3680 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e067fc700 1 -- 172.21.15.118:0/4154769105 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00007320 con 0x7f1e100c3680 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e101655b0 con 0x7f1e100c3680 2024-08-18T13:44:06.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.957+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e101657d0 con 0x7f1e100c3680 2024-08-18T13:44:06.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.961+0000 7f1e067fc700 1 -- 172.21.15.118:0/4154769105 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1e00007690 con 0x7f1e100c3680 2024-08-18T13:44:06.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.961+0000 7f1e067fc700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1df4060860 0x7f1df4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:06.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.961+0000 7f1e067fc700 1 -- 172.21.15.118:0/4154769105 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6801+0+0 (secure 0 0 0) 0x7f1e0007c490 con 0x7f1e100c3680 2024-08-18T13:44:06.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.961+0000 7f1e0dd9b700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1df4060860 0x7f1df4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:06.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.961+0000 7f1e0dd9b700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1df4060860 0x7f1df4062d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1df80093f0 tx=0x7f1df8017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:06.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:06.969+0000 7f1e067fc700 1 -- 172.21.15.118:0/4154769105 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1e0004c3f0 con 0x7f1e100c3680 2024-08-18T13:44:07.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.173+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e100713a0 con 0x7f1e100c3680 2024-08-18T13:44:07.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.173+0000 7f1e067fc700 1 -- 172.21.15.118:0/4154769105 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f1e0004fa10 con 0x7f1e100c3680 2024-08-18T13:44:07.178 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:07.179 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:07.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1df4060860 msgr2=0x7f1df4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:07.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1df4060860 0x7f1df4062d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1df80093f0 tx=0x7f1df8017000 comp rx=0 tx=0).stop 2024-08-18T13:44:07.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e100c3680 msgr2=0x7f1e100bfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:07.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e100c3680 0x7f1e100bfa80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e000011e0 tx=0x7f1e00007df0 comp rx=0 tx=0).stop 2024-08-18T13:44:07.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 shutdown_connections 2024-08-18T13:44:07.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1df4060860 0x7f1df4062d10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:07.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e100c3e40 0x7f1e10168360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:07.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e100c3a50 0x7f1e10158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:07.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 --2- 172.21.15.118:0/4154769105 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e100c3680 0x7f1e100bfa80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:07.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 >> 172.21.15.118:0/4154769105 conn(0x7f1e1006c020 msgr2=0x7f1e1014b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:07.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 shutdown_connections 2024-08-18T13:44:07.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:07.177+0000 7f1e0f59e700 1 -- 172.21.15.118:0/4154769105 wait complete. 2024-08-18T13:44:07.184 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:44:07.199 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:44:09.200 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:09.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4161ade700 1 Processor -- start 2024-08-18T13:44:09.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4161ade700 1 -- start start 2024-08-18T13:44:09.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4161ade700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c0c8200 0x7f415c0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:09.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4161ade700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f415c06f2f0 0x7f415c0c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:09.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4161ade700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c06f770 con 0x7f415c153690 2024-08-18T13:44:09.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4161ade700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c15a640 con 0x7f415c06f2f0 2024-08-18T13:44:09.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4161ade700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c06ee60 con 0x7f415c0c8200 2024-08-18T13:44:09.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4160adc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f415c153690 0x7f415c0c22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37124/0 (socket says 172.21.15.118:37124) 2024-08-18T13:44:09.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f4160adc700 1 -- 172.21.15.118:0/3998813250 learned_addr learned my addr 172.21.15.118:0/3998813250 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:09.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f41612dd700 1 --2- 172.21.15.118:0/3998813250 >> v2:172.21.15.118:3300/0 conn(0x7f415c06f2f0 0x7f415c0c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f415bfff700 1 --2- 172.21.15.118:0/3998813250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c0c8200 0x7f415c0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f415b7fe700 1 -- 172.21.15.118:0/3998813250 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387222561 0 0) 0x7f414c001490 con 0x7f415c153690 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f415b7fe700 1 -- 172.21.15.118:0/3998813250 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f413c003570 con 0x7f415c153690 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f41612dd700 1 -- 172.21.15.118:0/3998813250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c0c8200 msgr2=0x7f415c0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f41612dd700 1 --2- 172.21.15.118:0/3998813250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c0c8200 0x7f415c0673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f41612dd700 1 -- 172.21.15.118:0/3998813250 >> v1:172.21.15.118:6789/0 conn(0x7f415c153690 legacy=0x7f415c0c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f41612dd700 1 -- 172.21.15.118:0/3998813250 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c153d10 con 0x7f415c06f2f0 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f415bfff700 1 --2- 172.21.15.118:0/3998813250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c0c8200 0x7f415c0673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f41612dd700 1 --2- 172.21.15.118:0/3998813250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c06f2f0 0x7f415c0c29b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f41540098e0 tx=0x7f41540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fc86bdb5f388a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:09.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f415b7fe700 1 -- 172.21.15.118:0/3998813250 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154011630 con 0x7f415c06f2f0 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f415b7fe700 1 -- 172.21.15.118:0/3998813250 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154011790 con 0x7f415c06f2f0 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.309+0000 7f415b7fe700 1 -- 172.21.15.118:0/3998813250 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154011b10 con 0x7f415c06f2f0 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- 172.21.15.118:0/3998813250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c06f2f0 msgr2=0x7f415c0c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 --2- 172.21.15.118:0/3998813250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c06f2f0 0x7f415c0c29b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f41540098e0 tx=0x7f41540091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- 172.21.15.118:0/3998813250 shutdown_connections 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 --2- 172.21.15.118:0/3998813250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c06f2f0 0x7f415c0c29b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 --2- 172.21.15.118:0/3998813250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c0c8200 0x7f415c0673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- 172.21.15.118:0/3998813250 >> 172.21.15.118:0/3998813250 conn(0x7f415c06c020 msgr2=0x7f415c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- 172.21.15.118:0/3998813250 shutdown_connections 2024-08-18T13:44:09.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- 172.21.15.118:0/3998813250 wait complete. 2024-08-18T13:44:09.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 Processor -- start 2024-08-18T13:44:09.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- start start 2024-08-18T13:44:09.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c06f2f0 0x7f415c067b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c0c8200 0x7f415c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c153690 0x7f415c16d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c156f70 con 0x7f415c0c8200 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c16aa70 con 0x7f415c153690 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4161ade700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c16a5e0 con 0x7f415c06f2f0 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f415bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c0c8200 0x7f415c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f415bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c0c8200 0x7f415c15ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58234/0 (socket says 172.21.15.118:58234) 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f415bfff700 1 -- 172.21.15.118:0/1863028170 learned_addr learned my addr 172.21.15.118:0/1863028170 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.313+0000 7f4160adc700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c06f2f0 0x7f415c067b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f41612dd700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c153690 0x7f415c16d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f415bfff700 1 -- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c06f2f0 msgr2=0x7f415c067b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:09.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f415bfff700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c06f2f0 0x7f415c067b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f415bfff700 1 -- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c153690 msgr2=0x7f415c16d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f415bfff700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c153690 0x7f415c16d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f415bfff700 1 -- 172.21.15.118:0/1863028170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4144009420 con 0x7f415c0c8200 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f415bfff700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c0c8200 0x7f415c15ba30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f414400ecb0 tx=0x7f41440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4158ff9700 1 -- 172.21.15.118:0/1863028170 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400c9e0 con 0x7f415c0c8200 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4158ff9700 1 -- 172.21.15.118:0/1863028170 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414400cb40 con 0x7f415c0c8200 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4154009420 con 0x7f415c0c8200 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4158ff9700 1 -- 172.21.15.118:0/1863028170 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400cec0 con 0x7f415c0c8200 2024-08-18T13:44:09.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c169cb0 con 0x7f415c0c8200 2024-08-18T13:44:09.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c16a790 con 0x7f415c0c8200 2024-08-18T13:44:09.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4158ff9700 1 -- 172.21.15.118:0/1863028170 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4144013680 con 0x7f415c0c8200 2024-08-18T13:44:09.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4158ff9700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4140060860 0x7f4140062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:09.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4158ff9700 1 -- 172.21.15.118:0/1863028170 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6801+0+0 (secure 0 0 0) 0x7f414407d430 con 0x7f415c0c8200 2024-08-18T13:44:09.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.317+0000 7f4160adc700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4140060860 0x7f4140062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:09.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.321+0000 7f4160adc700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4140060860 0x7f4140062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f414c0015f0 tx=0x7f414c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:09.332 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.325+0000 7f4158ff9700 1 -- 172.21.15.118:0/1863028170 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f415c16a790 con 0x7f415c0c8200 2024-08-18T13:44:09.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.525+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f415c16a790 con 0x7f415c0c8200 2024-08-18T13:44:09.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4158ff9700 1 -- 172.21.15.118:0/1863028170 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f415c16a790 con 0x7f415c0c8200 2024-08-18T13:44:09.533 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:09.534 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4140060860 msgr2=0x7f4140062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4140060860 0x7f4140062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f414c0015f0 tx=0x7f414c009040 comp rx=0 tx=0).stop 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c0c8200 msgr2=0x7f415c15ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c0c8200 0x7f415c15ba30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f414400ecb0 tx=0x7f41440091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 shutdown_connections 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4140060860 0x7f4140062d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f415c153690 0x7f415c16d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f415c0c8200 0x7f415c15ba30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 --2- 172.21.15.118:0/1863028170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f415c06f2f0 0x7f415c067b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:09.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 >> 172.21.15.118:0/1863028170 conn(0x7f415c06c020 msgr2=0x7f415c06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:09.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 shutdown_connections 2024-08-18T13:44:09.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:09.529+0000 7f4161ade700 1 -- 172.21.15.118:0/1863028170 wait complete. 2024-08-18T13:44:09.537 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:44:09.547 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:44:11.549 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:11.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.657+0000 7f0d97fff700 1 Processor -- start 2024-08-18T13:44:11.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.657+0000 7f0d97fff700 1 -- start start 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 0x7f0d9815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d97fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d980c1110 con 0x7f0d980684e0 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d97fff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d980c1cb0 con 0x7f0d981556f0 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d97fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980c0b90 con 0x7f0d981561d0 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d8e7fc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f0d980684e0 0x7f0d98155ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37134/0 (socket says 172.21.15.118:37134) 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d8e7fc700 1 -- 172.21.15.118:0/3454669906 learned_addr learned my addr 172.21.15.118:0/3454669906 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d977fe700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 0x7f0d9815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460118566 0 0) 0x7f0d980c1110 con 0x7f0d980684e0 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d84003570 con 0x7f0d980684e0 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4176041135 0 0) 0x7f0d84003570 con 0x7f0d980684e0 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 msgr2=0x7f0d9815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:11.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 0x7f0d9815cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 >> v1:172.21.15.178:6789/0 conn(0x7f0d981556f0 legacy=0x7f0d980c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980c8060 con 0x7f0d980684e0 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f0d80001470 con 0x7f0d980684e0 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 >> v1:172.21.15.118:6789/0 conn(0x7f0d980684e0 legacy=0x7f0d98155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d84003a90 0x7f0d84003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 0x7f0d84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d84004d50 0x7f0d840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d980c0b90 con 0x7f0d84003a90 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8400bb40 con 0x7f0d84004d50 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8400bd00 con 0x7f0d981561d0 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d96ffd700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d84003a90 0x7f0d84003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d8e7fc700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 0x7f0d84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d977fe700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d84004d50 0x7f0d840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d96ffd700 1 -- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 msgr2=0x7f0d84004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d96ffd700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d981561d0 0x7f0d84004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d96ffd700 1 -- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d84004d50 msgr2=0x7f0d840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d96ffd700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d84004d50 0x7f0d840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d96ffd700 1 -- 172.21.15.118:0/3454669906 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d84003570 con 0x7f0d84003a90 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d977fe700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d84004d50 0x7f0d840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d96ffd700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d84003a90 0x7f0d84003e80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d88001970 tx=0x7f0d88007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad9a295c11e3c27 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800c5d0 con 0x7f0d84003a90 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8800c730 con 0x7f0d84003a90 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.661+0000 7f0d967fc700 1 -- 172.21.15.118:0/3454669906 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800cad0 con 0x7f0d84003a90 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- 172.21.15.118:0/3454669906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d84003a90 msgr2=0x7f0d84003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 --2- 172.21.15.118:0/3454669906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d84003a90 0x7f0d84003e80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d88001970 tx=0x7f0d88007d10 comp rx=0 tx=0).stop 2024-08-18T13:44:11.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d977fe700 1 -- 172.21.15.118:0/3454669906 reap_dead start 2024-08-18T13:44:11.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- 172.21.15.118:0/3454669906 shutdown_connections 2024-08-18T13:44:11.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- 172.21.15.118:0/3454669906 >> 172.21.15.118:0/3454669906 conn(0x7f0d9806c020 msgr2=0x7f0d980c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:11.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- 172.21.15.118:0/3454669906 shutdown_connections 2024-08-18T13:44:11.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- 172.21.15.118:0/3454669906 wait complete. 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 Processor -- start 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- start start 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9815a0e0 0x7f0d98163140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981672e0 0x7f0d980673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d9815a4b0 0x7f0d98067910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98143960 con 0x7f0d9815a4b0 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98166840 con 0x7f0d981672e0 2024-08-18T13:44:11.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.665+0000 7f0d97fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d98143c20 con 0x7f0d9815a0e0 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d9815a4b0 0x7f0d98067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d9815a4b0 0x7f0d98067910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58246/0 (socket says 172.21.15.118:58246) 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 -- 172.21.15.118:0/3993265981 learned_addr learned my addr 172.21.15.118:0/3993265981 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d96ffd700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9815a0e0 0x7f0d98163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d8e7fc700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981672e0 0x7f0d980673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 -- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9815a0e0 msgr2=0x7f0d98163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9815a0e0 0x7f0d98163140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 -- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981672e0 msgr2=0x7f0d980673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981672e0 0x7f0d980673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 -- 172.21.15.118:0/3993265981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98166b00 con 0x7f0d9815a4b0 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d977fe700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d9815a4b0 0x7f0d98067910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806c870 tx=0x7f0d9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d8f7fe700 1 -- 172.21.15.118:0/3993265981 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000fb00 con 0x7f0d9815a4b0 2024-08-18T13:44:11.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d98166de0 con 0x7f0d9815a4b0 2024-08-18T13:44:11.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d8f7fe700 1 -- 172.21.15.118:0/3993265981 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9000fc60 con 0x7f0d9815a4b0 2024-08-18T13:44:11.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d98149cf0 con 0x7f0d9815a4b0 2024-08-18T13:44:11.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d8f7fe700 1 -- 172.21.15.118:0/3993265981 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90016420 con 0x7f0d9815a4b0 2024-08-18T13:44:11.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d98166f70 con 0x7f0d9815a4b0 2024-08-18T13:44:11.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d8f7fe700 1 -- 172.21.15.118:0/3993265981 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0d90016580 con 0x7f0d9815a4b0 2024-08-18T13:44:11.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d8f7fe700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0d78060890 0x7f0d78062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:11.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.669+0000 7f0d8f7fe700 1 -- 172.21.15.118:0/3993265981 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6801+0+0 (secure 0 0 0) 0x7f0d9007d690 con 0x7f0d9815a4b0 2024-08-18T13:44:11.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.673+0000 7f0d96ffd700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0d78060890 0x7f0d78062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:11.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.673+0000 7f0d96ffd700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0d78060890 0x7f0d78062d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d98065b10 tx=0x7f0d88015000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:11.683 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.677+0000 7f0d8f7fe700 1 -- 172.21.15.118:0/3993265981 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0d9004d690 con 0x7f0d9815a4b0 2024-08-18T13:44:11.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.877+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d980713a0 con 0x7f0d9815a4b0 2024-08-18T13:44:11.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d8f7fe700 1 -- 172.21.15.118:0/3993265981 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f0d90050cb0 con 0x7f0d9815a4b0 2024-08-18T13:44:11.885 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:11.885 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0d78060890 msgr2=0x7f0d78062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0d78060890 0x7f0d78062d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0d98065b10 tx=0x7f0d88015000 comp rx=0 tx=0).stop 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d9815a4b0 msgr2=0x7f0d98067910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d9815a4b0 0x7f0d98067910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0d9806c870 tx=0x7f0d9000b040 comp rx=0 tx=0).stop 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 shutdown_connections 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0d78060890 0x7f0d78062d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0d9815a4b0 0x7f0d98067910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d981672e0 0x7f0d980673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 --2- 172.21.15.118:0/3993265981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9815a0e0 0x7f0d98163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:11.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 >> 172.21.15.118:0/3993265981 conn(0x7f0d9806c020 msgr2=0x7f0d98147f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:11.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 shutdown_connections 2024-08-18T13:44:11.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:11.881+0000 7f0d97fff700 1 -- 172.21.15.118:0/3993265981 wait complete. 2024-08-18T13:44:11.889 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:44:11.900 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:44:12.953 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:44:12.956 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:44:13.901 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:14.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50fcdff700 1 Processor -- start 2024-08-18T13:44:14.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50fcdff700 1 -- start start 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50fcdff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f50f80bd580 0x7f50f806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50fcdff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50fcdff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f8151f80 con 0x7f50f814bcd0 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50fcdff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f8151990 con 0x7f50f80bd580 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50fcdff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f814c0a0 con 0x7f50f80bea90 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f5d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f50f80bd580 0x7f50f806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f659c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f50f814bcd0 0x7f50f8152410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36426/0 (socket says 172.21.15.118:36426) 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f659c700 1 -- 172.21.15.118:0/2127180525 learned_addr learned my addr 172.21.15.118:0/2127180525 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:14.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f6d9d700 1 --2- 172.21.15.118:0/2127180525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:14.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f559a700 1 -- 172.21.15.118:0/2127180525 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651160952 0 0) 0x7f50f8151f80 con 0x7f50f814bcd0 2024-08-18T13:44:14.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f559a700 1 -- 172.21.15.118:0/2127180525 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e4003570 con 0x7f50f814bcd0 2024-08-18T13:44:14.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f5d9b700 1 -- 172.21.15.118:0/2127180525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 msgr2=0x7f50f806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:14.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f5d9b700 1 --2- 172.21.15.118:0/2127180525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f806e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f5d9b700 1 -- 172.21.15.118:0/2127180525 >> v1:172.21.15.118:6789/0 conn(0x7f50f814bcd0 legacy=0x7f50f8152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:14.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.009+0000 7f50f5d9b700 1 -- 172.21.15.118:0/2127180525 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80c2a50 con 0x7f50f80bd580 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50f5d9b700 1 --2- 172.21.15.118:0/2127180525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f806e290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009e60 tx=0x7f50e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07462168fd9a6c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50f559a700 1 -- 172.21.15.118:0/2127180525 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800e070 con 0x7f50f80bd580 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50f559a700 1 -- 172.21.15.118:0/2127180525 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e800bb40 con 0x7f50f80bd580 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50f559a700 1 -- 172.21.15.118:0/2127180525 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8012430 con 0x7f50f80bd580 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- 172.21.15.118:0/2127180525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 msgr2=0x7f50f806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 --2- 172.21.15.118:0/2127180525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f806e290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009e60 tx=0x7f50e8009780 comp rx=0 tx=0).stop 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- 172.21.15.118:0/2127180525 shutdown_connections 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 --2- 172.21.15.118:0/2127180525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f806e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 --2- 172.21.15.118:0/2127180525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f806e290 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- 172.21.15.118:0/2127180525 >> 172.21.15.118:0/2127180525 conn(0x7f50f80c7c40 msgr2=0x7f50f80c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- 172.21.15.118:0/2127180525 shutdown_connections 2024-08-18T13:44:14.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- 172.21.15.118:0/2127180525 wait complete. 2024-08-18T13:44:14.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 Processor -- start 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- start start 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f8149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f80c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f814bcd0 0x7f50f80c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f813f400 con 0x7f50f80bd580 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f813d2c0 con 0x7f50f814bcd0 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50fcdff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f813d7f0 con 0x7f50f80bea90 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50f659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f8149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50f659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f8149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33576/0 (socket says 172.21.15.118:33576) 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.013+0000 7f50f659c700 1 -- 172.21.15.118:0/962962360 learned_addr learned my addr 172.21.15.118:0/962962360 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:14.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f6d9d700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f814bcd0 0x7f50f80c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f5d9b700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f80c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f659c700 1 -- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 msgr2=0x7f50f80c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f659c700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f80c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f659c700 1 -- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f814bcd0 msgr2=0x7f50f80c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f659c700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f814bcd0 0x7f50f80c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f659c700 1 -- 172.21.15.118:0/962962360 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8009a20 con 0x7f50f80bd580 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50f659c700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f8149e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c7f0 tx=0x7f50e0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50ee7fc700 1 -- 172.21.15.118:0/962962360 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0005b30 con 0x7f50f80bd580 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50ee7fc700 1 -- 172.21.15.118:0/962962360 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0005c90 con 0x7f50f80bd580 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50ee7fc700 1 -- 172.21.15.118:0/962962360 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0007340 con 0x7f50f80bd580 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f813ef70 con 0x7f50f80bd580 2024-08-18T13:44:14.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f8149570 con 0x7f50f80bd580 2024-08-18T13:44:14.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f80642f0 con 0x7f50f80bd580 2024-08-18T13:44:14.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50ee7fc700 1 -- 172.21.15.118:0/962962360 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f50e001b400 con 0x7f50f80bd580 2024-08-18T13:44:14.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50ee7fc700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f50d8060890 0x7f50d8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:14.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.017+0000 7f50ee7fc700 1 -- 172.21.15.118:0/962962360 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6801+0+0 (secure 0 0 0) 0x7f50e007cb50 con 0x7f50f80bd580 2024-08-18T13:44:14.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.021+0000 7f50f5d9b700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f50d8060890 0x7f50d8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:14.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.021+0000 7f50f5d9b700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f50d8060890 0x7f50d8062d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097c0 tx=0x7f50e8019000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:14.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.025+0000 7f50ee7fc700 1 -- 172.21.15.118:0/962962360 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f50e004cbd0 con 0x7f50f80bd580 2024-08-18T13:44:14.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.225+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50f80bd080 con 0x7f50f80bd580 2024-08-18T13:44:14.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50ee7fc700 1 -- 172.21.15.118:0/962962360 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f50e00501f0 con 0x7f50f80bd580 2024-08-18T13:44:14.234 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:14.234 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:42:39.683128+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"f","rank":0,"incarnation":10,"state":"up:active","state_seq":51,"addr":"172.21.15.178:6835/1590569622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1590569622},{"type":"v1","addr":"172.21.15.178:6835","nonce":1590569622}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:14.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f50d8060890 msgr2=0x7f50d8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:14.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f50d8060890 0x7f50d8062d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097c0 tx=0x7f50e8019000 comp rx=0 tx=0).stop 2024-08-18T13:44:14.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 msgr2=0x7f50f8149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:14.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f8149e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c7f0 tx=0x7f50e0007e00 comp rx=0 tx=0).stop 2024-08-18T13:44:14.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 shutdown_connections 2024-08-18T13:44:14.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f50d8060890 0x7f50d8062d40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50f814bcd0 0x7f50f80c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f50f80bea90 0x7f50f80c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 --2- 172.21.15.118:0/962962360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50f80bd580 0x7f50f8149e60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:14.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 >> 172.21.15.118:0/962962360 conn(0x7f50f80c7c40 msgr2=0x7f50f806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:14.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 shutdown_connections 2024-08-18T13:44:14.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:14.229+0000 7f50fcdff700 1 -- 172.21.15.118:0/962962360 wait complete. 2024-08-18T13:44:14.238 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 15 2024-08-18T13:44:14.248 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:44:16.249 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:16.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 Processor -- start 2024-08-18T13:44:16.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 -- start start 2024-08-18T13:44:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f67b40684e0 0x7f67b4155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b40c1110 con 0x7f67b40be4b0 2024-08-18T13:44:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b40c1cb0 con 0x7f67b41556f0 2024-08-18T13:44:16.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b40c0b90 con 0x7f67b40684e0 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67ba35b700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f67b40be4b0 0x7f67b4151f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36438/0 (socket says 172.21.15.118:36438) 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67ba35b700 1 -- 172.21.15.118:0/695637104 learned_addr learned my addr 172.21.15.118:0/695637104 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b9359700 1 --2- 172.21.15.118:0/695637104 >> v2:172.21.15.178:3300/0 conn(0x7f67b40684e0 0x7f67b4155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b8b58700 1 -- 172.21.15.118:0/695637104 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834071347 0 0) 0x7f67b40c1110 con 0x7f67b40be4b0 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b8b58700 1 -- 172.21.15.118:0/695637104 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a0003570 con 0x7f67b40be4b0 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b9359700 1 -- 172.21.15.118:0/695637104 >> v1:172.21.15.178:6789/0 conn(0x7f67b41556f0 legacy=0x7f67b40c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b9359700 1 -- 172.21.15.118:0/695637104 >> v1:172.21.15.118:6789/0 conn(0x7f67b40be4b0 legacy=0x7f67b4151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b9359700 1 -- 172.21.15.118:0/695637104 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40c8060 con 0x7f67b40684e0 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b9359700 1 --2- 172.21.15.118:0/695637104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40684e0 0x7f67b4155ac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009e60 tx=0x7f67ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c51ea3f3a8daa90 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b8b58700 1 -- 172.21.15.118:0/695637104 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00e070 con 0x7f67b40684e0 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b8b58700 1 -- 172.21.15.118:0/695637104 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00bb60 con 0x7f67b40684e0 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67b8b58700 1 -- 172.21.15.118:0/695637104 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac012430 con 0x7f67b40684e0 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 -- 172.21.15.118:0/695637104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40684e0 msgr2=0x7f67b4155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 --2- 172.21.15.118:0/695637104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40684e0 0x7f67b4155ac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009e60 tx=0x7f67ac009780 comp rx=0 tx=0).stop 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 -- 172.21.15.118:0/695637104 shutdown_connections 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 --2- 172.21.15.118:0/695637104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40684e0 0x7f67b4155ac0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.361+0000 7f67bab5c700 1 -- 172.21.15.118:0/695637104 >> 172.21.15.118:0/695637104 conn(0x7f67b406c020 msgr2=0x7f67b40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- 172.21.15.118:0/695637104 shutdown_connections 2024-08-18T13:44:16.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- 172.21.15.118:0/695637104 wait complete. 2024-08-18T13:44:16.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 Processor -- start 2024-08-18T13:44:16.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- start start 2024-08-18T13:44:16.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67b40684e0 0x7f67b41512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40be4b0 0x7f67b4168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67b41556f0 0x7f67b4147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4169140 con 0x7f67b41556f0 2024-08-18T13:44:16.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40c3280 con 0x7f67b40be4b0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4148620 con 0x7f67b40684e0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67b41556f0 0x7f67b4147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67b41556f0 0x7f67b4147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33580/0 (socket says 172.21.15.118:33580) 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 -- 172.21.15.118:0/3409192996 learned_addr learned my addr 172.21.15.118:0/3409192996 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67b9359700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40be4b0 0x7f67b4168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67b9b5a700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67b40684e0 0x7f67b41512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 -- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67b40684e0 msgr2=0x7f67b41512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67b40684e0 0x7f67b41512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 -- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40be4b0 msgr2=0x7f67b4168470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40be4b0 0x7f67b4168470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 -- 172.21.15.118:0/3409192996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac009a20 con 0x7f67b41556f0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67ba35b700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67b41556f0 0x7f67b4147890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f67b00010e0 tx=0x7f67b0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67a5ffb700 1 -- 172.21.15.118:0/3409192996 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0005c00 con 0x7f67b41556f0 2024-08-18T13:44:16.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67a5ffb700 1 -- 172.21.15.118:0/3409192996 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0005d60 con 0x7f67b41556f0 2024-08-18T13:44:16.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67a5ffb700 1 -- 172.21.15.118:0/3409192996 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00072e0 con 0x7f67b41556f0 2024-08-18T13:44:16.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b4168e60 con 0x7f67b41556f0 2024-08-18T13:44:16.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.365+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b406f2c0 con 0x7f67b41556f0 2024-08-18T13:44:16.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.369+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b406eba0 con 0x7f67b41556f0 2024-08-18T13:44:16.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.369+0000 7f67a5ffb700 1 -- 172.21.15.118:0/3409192996 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f67b0012400 con 0x7f67b41556f0 2024-08-18T13:44:16.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.369+0000 7f67a5ffb700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6798060860 0x7f6798062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.369+0000 7f67a5ffb700 1 -- 172.21.15.118:0/3409192996 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f67b007c4f0 con 0x7f67b41556f0 2024-08-18T13:44:16.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.369+0000 7f67b9b5a700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6798060860 0x7f6798062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.369+0000 7f67b9b5a700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6798060860 0x7f6798062d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f67a8003880 tx=0x7f67a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:16.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.377+0000 7f67a5ffb700 1 -- 172.21.15.118:0/3409192996 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f67b406eba0 con 0x7f67b41556f0 2024-08-18T13:44:16.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.577+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67b406eba0 con 0x7f67b41556f0 2024-08-18T13:44:16.583 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.577+0000 7f67a5ffb700 1 -- 172.21.15.118:0/3409192996 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f67b406eba0 con 0x7f67b41556f0 2024-08-18T13:44:16.583 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:16.583 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:15.545290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6798060860 msgr2=0x7f6798062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6798060860 0x7f6798062d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f67a8003880 tx=0x7f67a8009040 comp rx=0 tx=0).stop 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67b41556f0 msgr2=0x7f67b4147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67b41556f0 0x7f67b4147890 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f67b00010e0 tx=0x7f67b0007dd0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 shutdown_connections 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6798060860 0x7f6798062d10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f67b41556f0 0x7f67b4147890 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67b40be4b0 0x7f67b4168470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 --2- 172.21.15.118:0/3409192996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67b40684e0 0x7f67b41512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 >> 172.21.15.118:0/3409192996 conn(0x7f67b406c020 msgr2=0x7f67b40673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:16.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 shutdown_connections 2024-08-18T13:44:16.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.581+0000 7f67bab5c700 1 -- 172.21.15.118:0/3409192996 wait complete. 2024-08-18T13:44:16.587 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 16 2024-08-18T13:44:16.602 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-08-18T13:44:16.602 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-08-18T13:44:16.602 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:16.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 Processor -- start 2024-08-18T13:44:16.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- start start 2024-08-18T13:44:16.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2c0c14b940 0x7f2c0c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.714 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2c0c14bd10 0x7f2c0c151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c06fac0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c151870 con 0x7f2c0c14b940 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c06de00 con 0x7f2c0c0be5a0 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2c0c14bd10 0x7f2c0c151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2c0c14bd10 0x7f2c0c151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33590/0 (socket says 172.21.15.118:33590) 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 -- 172.21.15.118:0/3728888784 learned_addr learned my addr 172.21.15.118:0/3728888784 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c116b8700 1 --2- 172.21.15.118:0/3728888784 >> v2:172.21.15.178:3300/0 conn(0x7f2c0c14b940 0x7f2c0c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c10eb7700 1 --2- 172.21.15.118:0/3728888784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 -- 172.21.15.118:0/3728888784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 msgr2=0x7f2c0c06ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 --2- 172.21.15.118:0/3728888784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c06ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 -- 172.21.15.118:0/3728888784 >> v2:172.21.15.178:3300/0 conn(0x7f2c0c14b940 msgr2=0x7f2c0c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 --2- 172.21.15.118:0/3728888784 >> v2:172.21.15.178:3300/0 conn(0x7f2c0c14b940 0x7f2c0c06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.715 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 -- 172.21.15.118:0/3728888784 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0c2da0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c10eb7700 1 --2- 172.21.15.118:0/3728888784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c06ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:16.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c116b8700 1 --2- 172.21.15.118:0/3728888784 >> v2:172.21.15.178:3300/0 conn(0x7f2c0c14b940 0x7f2c0c06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:16.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c11eb9700 1 --2- 172.21.15.118:0/3728888784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c151cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c06a920 tx=0x7f2c04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134c481a9cf2bc9b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:16.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c03fff700 1 -- 172.21.15.118:0/3728888784 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0400b860 con 0x7f2c0c14bd10 2024-08-18T13:44:16.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c03fff700 1 -- 172.21.15.118:0/3728888784 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0400b9c0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c03fff700 1 -- 172.21.15.118:0/3728888784 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0400bd60 con 0x7f2c0c14bd10 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- 172.21.15.118:0/3728888784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 msgr2=0x7f2c0c151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 --2- 172.21.15.118:0/3728888784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c151cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c06a920 tx=0x7f2c04009250 comp rx=0 tx=0).stop 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- 172.21.15.118:0/3728888784 shutdown_connections 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 --2- 172.21.15.118:0/3728888784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c151cc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 --2- 172.21.15.118:0/3728888784 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c06ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 --2- 172.21.15.118:0/3728888784 >> v2:172.21.15.178:3300/0 conn(0x7f2c0c14b940 0x7f2c0c06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- 172.21.15.118:0/3728888784 >> 172.21.15.118:0/3728888784 conn(0x7f2c0c06c020 msgr2=0x7f2c0c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- 172.21.15.118:0/3728888784 shutdown_connections 2024-08-18T13:44:16.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.709+0000 7f2c126ba700 1 -- 172.21.15.118:0/3728888784 wait complete. 2024-08-18T13:44:16.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 Processor -- start 2024-08-18T13:44:16.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 -- start start 2024-08-18T13:44:16.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c0c14b940 0x7f2c0c1467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c1491c0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c146350 con 0x7f2c0c14b940 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c126ba700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c13d040 con 0x7f2c0c0be5a0 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c11eb9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c11eb9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c146d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33594/0 (socket says 172.21.15.118:33594) 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c11eb9700 1 -- 172.21.15.118:0/40637026 learned_addr learned my addr 172.21.15.118:0/40637026 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c10eb7700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c0c14b940 0x7f2c0c1467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.713+0000 7f2c116b8700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c11eb9700 1 -- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 msgr2=0x7f2c0c140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c11eb9700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c11eb9700 1 -- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c0c14b940 msgr2=0x7f2c0c1467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c11eb9700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c0c14b940 0x7f2c0c1467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c11eb9700 1 -- 172.21.15.118:0/40637026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c040094f0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c11eb9700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c146d20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c05ea40 tx=0x7f2c04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c017fa700 1 -- 172.21.15.118:0/40637026 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0401a700 con 0x7f2c0c14bd10 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c017fa700 1 -- 172.21.15.118:0/40637026 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0401a860 con 0x7f2c0c14bd10 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c148ee0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c017fa700 1 -- 172.21.15.118:0/40637026 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0401ac00 con 0x7f2c0c14bd10 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c145d00 con 0x7f2c0c14bd10 2024-08-18T13:44:16.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.717+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c067cd0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.721+0000 7f2c017fa700 1 -- 172.21.15.118:0/40637026 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2c040233e0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.721+0000 7f2c017fa700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bf8060890 0x7f2bf8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:16.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.721+0000 7f2c116b8700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bf8060890 0x7f2bf8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:16.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.721+0000 7f2c017fa700 1 -- 172.21.15.118:0/40637026 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f2c0407ed00 con 0x7f2c0c14bd10 2024-08-18T13:44:16.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.721+0000 7f2c116b8700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bf8060890 0x7f2bf8062d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc006fd0 tx=0x7f2bfc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:16.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.729+0000 7f2c017fa700 1 -- 172.21.15.118:0/40637026 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2c0c067cd0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.935 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.929+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c0c067cd0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.935 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.929+0000 7f2c017fa700 1 -- 172.21.15.118:0/40637026 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f2c0c067cd0 con 0x7f2c0c14bd10 2024-08-18T13:44:16.936 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:16.936 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:15.545290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:16.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bf8060890 msgr2=0x7f2bf8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bf8060890 0x7f2bf8062d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc006fd0 tx=0x7f2bfc008040 comp rx=0 tx=0).stop 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 msgr2=0x7f2c0c146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c146d20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c05ea40 tx=0x7f2c04017000 comp rx=0 tx=0).stop 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 shutdown_connections 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bf8060890 0x7f2bf8062d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2c0c14bd10 0x7f2c0c146d20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c0c14b940 0x7f2c0c1467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 --2- 172.21.15.118:0/40637026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c0c0be5a0 0x7f2c0c140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 >> 172.21.15.118:0/40637026 conn(0x7f2c0c06c020 msgr2=0x7f2c0c150580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 shutdown_connections 2024-08-18T13:44:16.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:16.933+0000 7f2c126ba700 1 -- 172.21.15.118:0/40637026 wait complete. 2024-08-18T13:44:16.940 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 16 2024-08-18T13:44:16.950 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 16, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4805": { "addr": "172.21.15.178:6837/3452743394", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6836", "nonce": 3452743394, "type": "v2" }, { "addr": "172.21.15.178:6837", "nonce": 3452743394, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4805, "incarnation": 16, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:44:15.545290+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4805 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:44:18.951 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:19.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f57307b4700 1 Processor -- start 2024-08-18T13:44:19.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f57307b4700 1 -- start start 2024-08-18T13:44:19.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f57307b4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f572814bcd0 0x7f5728152410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:19.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f57307b4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5728151f80 con 0x7f57280bd580 2024-08-18T13:44:19.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f57307b4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5728151990 con 0x7f57280bea90 2024-08-18T13:44:19.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f57307b4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572814c0a0 con 0x7f572814bcd0 2024-08-18T13:44:19.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572e550700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f572814bcd0 0x7f5728152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:19.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572e550700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f572814bcd0 0x7f5728152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33606/0 (socket says 172.21.15.118:33606) 2024-08-18T13:44:19.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572dd4f700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f57280bd580 0x7f572806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36448/0 (socket says 172.21.15.118:36448) 2024-08-18T13:44:19.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572e550700 1 -- 172.21.15.118:0/277363565 learned_addr learned my addr 172.21.15.118:0/277363565 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:19.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572e550700 1 -- 172.21.15.118:0/277363565 >> v1:172.21.15.178:6789/0 conn(0x7f57280bea90 legacy=0x7f572806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:19.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572e550700 1 -- 172.21.15.118:0/277363565 >> v1:172.21.15.118:6789/0 conn(0x7f57280bd580 legacy=0x7f572806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:19.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572e550700 1 -- 172.21.15.118:0/277363565 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728152950 con 0x7f572814bcd0 2024-08-18T13:44:19.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572e550700 1 --2- 172.21.15.118:0/277363565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f5728152410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5720009f20 tx=0x7f5720009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea72fc689c6a3254 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:19.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572d54e700 1 -- 172.21.15.118:0/277363565 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572000e070 con 0x7f572814bcd0 2024-08-18T13:44:19.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572d54e700 1 -- 172.21.15.118:0/277363565 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572000bad0 con 0x7f572814bcd0 2024-08-18T13:44:19.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.065+0000 7f572d54e700 1 -- 172.21.15.118:0/277363565 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57200092e0 con 0x7f572814bcd0 2024-08-18T13:44:19.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 -- 172.21.15.118:0/277363565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 msgr2=0x7f5728152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:19.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 --2- 172.21.15.118:0/277363565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f5728152410 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5720009f20 tx=0x7f5720009840 comp rx=0 tx=0).stop 2024-08-18T13:44:19.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 -- 172.21.15.118:0/277363565 shutdown_connections 2024-08-18T13:44:19.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 --2- 172.21.15.118:0/277363565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f5728152410 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:19.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 -- 172.21.15.118:0/277363565 >> 172.21.15.118:0/277363565 conn(0x7f57280c7c40 msgr2=0x7f57280c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:19.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 -- 172.21.15.118:0/277363565 shutdown_connections 2024-08-18T13:44:19.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 -- 172.21.15.118:0/277363565 wait complete. 2024-08-18T13:44:19.072 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 Processor -- start 2024-08-18T13:44:19.072 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.069+0000 7f57307b4700 1 -- start start 2024-08-18T13:44:19.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57280bd580 0x7f57280c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:19.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280bea90 0x7f57280c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:19.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f57280c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:19.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728146080 con 0x7f572814bcd0 2024-08-18T13:44:19.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572814ece0 con 0x7f57280bd580 2024-08-18T13:44:19.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57280c09c0 con 0x7f57280bea90 2024-08-18T13:44:19.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f57280c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:19.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f57280c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33620/0 (socket says 172.21.15.118:33620) 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 -- 172.21.15.118:0/684002612 learned_addr learned my addr 172.21.15.118:0/684002612 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572e550700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57280bd580 0x7f57280c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572dd4f700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280bea90 0x7f57280c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 -- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280bea90 msgr2=0x7f57280c1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280bea90 0x7f57280c1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 -- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57280bd580 msgr2=0x7f57280c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57280bd580 0x7f57280c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:19.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 -- 172.21.15.118:0/684002612 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720009ae0 con 0x7f572814bcd0 2024-08-18T13:44:19.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f572ed51700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f57280c6ef0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f572400c5b0 tx=0x7f5724007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:19.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f571e7fc700 1 -- 172.21.15.118:0/684002612 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724011590 con 0x7f572814bcd0 2024-08-18T13:44:19.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f571e7fc700 1 -- 172.21.15.118:0/684002612 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5724011710 con 0x7f572814bcd0 2024-08-18T13:44:19.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572814eed0 con 0x7f572814bcd0 2024-08-18T13:44:19.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f571e7fc700 1 -- 172.21.15.118:0/684002612 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724011ad0 con 0x7f572814bcd0 2024-08-18T13:44:19.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.073+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57280c0650 con 0x7f572814bcd0 2024-08-18T13:44:19.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.077+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572806c810 con 0x7f572814bcd0 2024-08-18T13:44:19.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.077+0000 7f571e7fc700 1 -- 172.21.15.118:0/684002612 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f572401c460 con 0x7f572814bcd0 2024-08-18T13:44:19.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.077+0000 7f571e7fc700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f570c060890 0x7f570c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:19.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.077+0000 7f571e7fc700 1 -- 172.21.15.118:0/684002612 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f572407cb60 con 0x7f572814bcd0 2024-08-18T13:44:19.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.077+0000 7f572e550700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f570c060890 0x7f570c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:19.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.077+0000 7f572e550700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f570c060890 0x7f570c062d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5720009880 tx=0x7f5720017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:19.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.085+0000 7f571e7fc700 1 -- 172.21.15.118:0/684002612 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f572806c810 con 0x7f572814bcd0 2024-08-18T13:44:19.287 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.285+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5728067580 con 0x7f572814bcd0 2024-08-18T13:44:19.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.285+0000 7f571e7fc700 1 -- 172.21.15.118:0/684002612 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f5728067580 con 0x7f572814bcd0 2024-08-18T13:44:19.289 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:19.289 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:15.545290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:19.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f570c060890 msgr2=0x7f570c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:19.291 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f570c060890 0x7f570c062d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5720009880 tx=0x7f5720017000 comp rx=0 tx=0).stop 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 msgr2=0x7f57280c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f57280c6ef0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f572400c5b0 tx=0x7f5724007f60 comp rx=0 tx=0).stop 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 shutdown_connections 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f570c060890 0x7f570c062d40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f572814bcd0 0x7f57280c6ef0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280bea90 0x7f57280c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 --2- 172.21.15.118:0/684002612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57280bd580 0x7f57280c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 >> 172.21.15.118:0/684002612 conn(0x7f57280c7c40 msgr2=0x7f5728146a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 shutdown_connections 2024-08-18T13:44:19.292 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:19.289+0000 7f57307b4700 1 -- 172.21.15.118:0/684002612 wait complete. 2024-08-18T13:44:19.293 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 16 2024-08-18T13:44:19.304 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:44:21.304 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:21.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.416+0000 7f75970a4700 1 Processor -- start 2024-08-18T13:44:21.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.416+0000 7f75970a4700 1 -- start start 2024-08-18T13:44:21.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f75900c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:21.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7590068e00 con 0x7f759006a080 2024-08-18T13:44:21.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75900690a0 con 0x7f75900c0f40 2024-08-18T13:44:21.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7590069340 con 0x7f75900c0b70 2024-08-18T13:44:21.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75968a3700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f759006a080 0x7f759014efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:36462/0 (socket says 172.21.15.118:36462) 2024-08-18T13:44:21.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75968a3700 1 -- 172.21.15.118:0/4111259985 learned_addr learned my addr 172.21.15.118:0/4111259985 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:21.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75960a2700 1 --2- 172.21.15.118:0/4111259985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f75900c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:21.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75950a0700 1 -- 172.21.15.118:0/4111259985 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575488635 0 0) 0x7f7590068e00 con 0x7f759006a080 2024-08-18T13:44:21.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75950a0700 1 -- 172.21.15.118:0/4111259985 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f757c003570 con 0x7f759006a080 2024-08-18T13:44:21.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75950a0700 1 -- 172.21.15.118:0/4111259985 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352997478 0 0) 0x7f75900690a0 con 0x7f75900c0f40 2024-08-18T13:44:21.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75960a2700 1 -- 172.21.15.118:0/4111259985 >> v1:172.21.15.178:6789/0 conn(0x7f75900c0f40 legacy=0x7f75900cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:21.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75960a2700 1 -- 172.21.15.118:0/4111259985 >> v1:172.21.15.118:6789/0 conn(0x7f759006a080 legacy=0x7f759014efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:21.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75960a2700 1 -- 172.21.15.118:0/4111259985 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900695e0 con 0x7f75900c0b70 2024-08-18T13:44:21.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75960a2700 1 --2- 172.21.15.118:0/4111259985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f75900c6510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7580009910 tx=0x7f75800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434319bac7912b00 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:21.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75950a0700 1 -- 172.21.15.118:0/4111259985 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758000b710 con 0x7f75900c0b70 2024-08-18T13:44:21.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75950a0700 1 -- 172.21.15.118:0/4111259985 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758000b870 con 0x7f75900c0b70 2024-08-18T13:44:21.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75950a0700 1 -- 172.21.15.118:0/4111259985 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758000bc30 con 0x7f75900c0b70 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- 172.21.15.118:0/4111259985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 msgr2=0x7f75900c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 --2- 172.21.15.118:0/4111259985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f75900c6510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7580009910 tx=0x7f75800091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- 172.21.15.118:0/4111259985 shutdown_connections 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 --2- 172.21.15.118:0/4111259985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f75900c6510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- 172.21.15.118:0/4111259985 >> 172.21.15.118:0/4111259985 conn(0x7f7590067e50 msgr2=0x7f75900cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- 172.21.15.118:0/4111259985 shutdown_connections 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.420+0000 7f75970a4700 1 -- 172.21.15.118:0/4111259985 wait complete. 2024-08-18T13:44:21.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 Processor -- start 2024-08-18T13:44:21.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 -- start start 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f759006a080 0x7f75900b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f7590165430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75900c0f40 0x7f7590165970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75900c4eb0 con 0x7f75900c0f40 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75900c3660 con 0x7f759006a080 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75900c38b0 con 0x7f75900c0b70 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75968a3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75900c0f40 0x7f7590165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75968a3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75900c0f40 0x7f7590165970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33634/0 (socket says 172.21.15.118:33634) 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75968a3700 1 -- 172.21.15.118:0/2800823816 learned_addr learned my addr 172.21.15.118:0/2800823816 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:21.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75958a1700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f7590165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75960a2700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f759006a080 0x7f75900b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75958a1700 1 -- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f759006a080 msgr2=0x7f75900b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75958a1700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f759006a080 0x7f75900b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75958a1700 1 -- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75900c0f40 msgr2=0x7f7590165970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75958a1700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75900c0f40 0x7f7590165970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75958a1700 1 -- 172.21.15.118:0/2800823816 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580009420 con 0x7f75900c0b70 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75968a3700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75900c0f40 0x7f7590165970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:44:21.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75958a1700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f7590165430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f758c00c540 tx=0x7f758c007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:21.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75867fc700 1 -- 172.21.15.118:0/2800823816 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c011590 con 0x7f75900c0b70 2024-08-18T13:44:21.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75867fc700 1 -- 172.21.15.118:0/2800823816 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c011710 con 0x7f75900c0b70 2024-08-18T13:44:21.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75900c3b90 con 0x7f75900c0b70 2024-08-18T13:44:21.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75867fc700 1 -- 172.21.15.118:0/2800823816 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c011a90 con 0x7f75900c0b70 2024-08-18T13:44:21.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.424+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75900c4150 con 0x7f75900c0b70 2024-08-18T13:44:21.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.428+0000 7f75867fc700 1 -- 172.21.15.118:0/2800823816 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f758c01b3b0 con 0x7f75900c0b70 2024-08-18T13:44:21.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.428+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75900713a0 con 0x7f75900c0b70 2024-08-18T13:44:21.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.428+0000 7f75867fc700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7574060890 0x7f7574062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:21.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.428+0000 7f75867fc700 1 -- 172.21.15.118:0/2800823816 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f758c07c330 con 0x7f75900c0b70 2024-08-18T13:44:21.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.428+0000 7f75960a2700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7574060890 0x7f7574062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:21.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.428+0000 7f75960a2700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7574060890 0x7f7574062d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f758000c010 tx=0x7f7580017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:21.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.440+0000 7f75867fc700 1 -- 172.21.15.118:0/2800823816 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f758c04c330 con 0x7f75900c0b70 2024-08-18T13:44:21.646 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.644+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75900697c0 con 0x7f75900c0b70 2024-08-18T13:44:21.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.644+0000 7f75867fc700 1 -- 172.21.15.118:0/2800823816 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f758c04f950 con 0x7f75900c0b70 2024-08-18T13:44:21.647 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:21.647 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:15.545290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7574060890 msgr2=0x7f7574062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7574060890 0x7f7574062d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f758000c010 tx=0x7f7580017000 comp rx=0 tx=0).stop 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 msgr2=0x7f7590165430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f7590165430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f758c00c540 tx=0x7f758c007ec0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 shutdown_connections 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7574060890 0x7f7574062d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75900c0f40 0x7f7590165970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75900c0b70 0x7f7590165430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 --2- 172.21.15.118:0/2800823816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f759006a080 0x7f75900b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 >> 172.21.15.118:0/2800823816 conn(0x7f7590067e50 msgr2=0x7f759014dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 shutdown_connections 2024-08-18T13:44:21.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:21.648+0000 7f75970a4700 1 -- 172.21.15.118:0/2800823816 wait complete. 2024-08-18T13:44:21.652 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 16 2024-08-18T13:44:21.665 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:44:23.666 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:23.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc32a622700 1 Processor -- start 2024-08-18T13:44:23.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc32a622700 1 -- start start 2024-08-18T13:44:23.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc32a622700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc324069680 0x7fc3240c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:23.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc32a622700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc324157200 0x7fc324153090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:23.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc32a622700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3240c95b0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc32a622700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3241566a0 con 0x7fc324157200 2024-08-18T13:44:23.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc32a622700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3240c9840 con 0x7fc324069680 2024-08-18T13:44:23.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc324157200 0x7fc324153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc324157200 0x7fc324153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33640/0 (socket says 172.21.15.118:33640) 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 -- 172.21.15.118:0/2005875764 learned_addr learned my addr 172.21.15.118:0/2005875764 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329620700 1 --2- 172.21.15.118:0/2005875764 >> v2:172.21.15.178:3300/0 conn(0x7fc324069680 0x7fc3240c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 -- 172.21.15.118:0/2005875764 >> v2:172.21.15.178:3300/0 conn(0x7fc324069680 msgr2=0x7fc3240c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 --2- 172.21.15.118:0/2005875764 >> v2:172.21.15.178:3300/0 conn(0x7fc324069680 0x7fc3240c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 -- 172.21.15.118:0/2005875764 >> v1:172.21.15.178:6789/0 conn(0x7fc3240c2bf0 legacy=0x7fc324156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 -- 172.21.15.118:0/2005875764 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc324067470 con 0x7fc324157200 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc329e21700 1 --2- 172.21.15.118:0/2005875764 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc324157200 0x7fc324153090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc32406db10 tx=0x7fc31c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25de02c80d99cff4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc31bfff700 1 -- 172.21.15.118:0/2005875764 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00e070 con 0x7fc324157200 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc31bfff700 1 -- 172.21.15.118:0/2005875764 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00ba20 con 0x7fc324157200 2024-08-18T13:44:23.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.776+0000 7fc31bfff700 1 -- 172.21.15.118:0/2005875764 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0092e0 con 0x7fc324157200 2024-08-18T13:44:23.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 -- 172.21.15.118:0/2005875764 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc324157200 msgr2=0x7fc324153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:23.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 --2- 172.21.15.118:0/2005875764 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc324157200 0x7fc324153090 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc32406db10 tx=0x7fc31c009840 comp rx=0 tx=0).stop 2024-08-18T13:44:23.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 -- 172.21.15.118:0/2005875764 shutdown_connections 2024-08-18T13:44:23.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 --2- 172.21.15.118:0/2005875764 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc324157200 0x7fc324153090 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:23.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 --2- 172.21.15.118:0/2005875764 >> v2:172.21.15.178:3300/0 conn(0x7fc324069680 0x7fc3240c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:23.782 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 -- 172.21.15.118:0/2005875764 >> 172.21.15.118:0/2005875764 conn(0x7fc32406f420 msgr2=0x7fc3240c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:23.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 -- 172.21.15.118:0/2005875764 shutdown_connections 2024-08-18T13:44:23.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 -- 172.21.15.118:0/2005875764 wait complete. 2024-08-18T13:44:23.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 Processor -- start 2024-08-18T13:44:23.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 -- start start 2024-08-18T13:44:23.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.780+0000 7fc32a622700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc324069680 0x7fc3241644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:23.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3240c2bf0 0x7fc324149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:23.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc324157200 0x7fc324163040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:23.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3241491d0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3241638f0 con 0x7fc324069680 2024-08-18T13:44:23.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc324163bd0 con 0x7fc324157200 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3240c2bf0 0x7fc324149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3240c2bf0 0x7fc324149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33656/0 (socket says 172.21.15.118:33656) 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc329620700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc324069680 0x7fc3241644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 -- 172.21.15.118:0/1398422416 learned_addr learned my addr 172.21.15.118:0/1398422416 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc329e21700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc324157200 0x7fc324163040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 -- 172.21.15.118:0/1398422416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc324157200 msgr2=0x7fc324163040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc324157200 0x7fc324163040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 -- 172.21.15.118:0/1398422416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc324069680 msgr2=0x7fc3241644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc324069680 0x7fc3241644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 -- 172.21.15.118:0/1398422416 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c009ae0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc328e1f700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3240c2bf0 0x7fc324149c70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c790 tx=0x7fc320007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc3197fa700 1 -- 172.21.15.118:0/1398422416 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320011590 con 0x7fc3240c2bf0 2024-08-18T13:44:23.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc3197fa700 1 -- 172.21.15.118:0/1398422416 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3200116f0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc3197fa700 1 -- 172.21.15.118:0/1398422416 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320011ab0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc324149470 con 0x7fc3240c2bf0 2024-08-18T13:44:23.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc324163eb0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3240713a0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.784+0000 7fc3197fa700 1 -- 172.21.15.118:0/1398422416 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc32001b3b0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.788+0000 7fc3197fa700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc308060860 0x7fc308062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:23.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.788+0000 7fc3197fa700 1 -- 172.21.15.118:0/1398422416 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fc32007c530 con 0x7fc3240c2bf0 2024-08-18T13:44:23.789 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.788+0000 7fc329620700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc308060860 0x7fc308062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:23.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.788+0000 7fc329620700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc308060860 0x7fc308062d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc314009db0 tx=0x7fc3140091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:23.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.792+0000 7fc3197fa700 1 -- 172.21.15.118:0/1398422416 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc32004c480 con 0x7fc3240c2bf0 2024-08-18T13:44:23.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.996+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc324158700 con 0x7fc3240c2bf0 2024-08-18T13:44:23.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:23.996+0000 7fc3197fa700 1 -- 172.21.15.118:0/1398422416 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5373 (secure 0 0 0) 0x7fc32004faa0 con 0x7fc3240c2bf0 2024-08-18T13:44:23.998 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:23.999 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:23.636905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:24.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc308060860 msgr2=0x7fc308062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:24.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc308060860 0x7fc308062d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc314009db0 tx=0x7fc3140091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:24.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3240c2bf0 msgr2=0x7fc324149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3240c2bf0 0x7fc324149c70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c790 tx=0x7fc320007f40 comp rx=0 tx=0).stop 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 shutdown_connections 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc308060860 0x7fc308062d10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc324157200 0x7fc324163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc3240c2bf0 0x7fc324149c70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 --2- 172.21.15.118:0/1398422416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc324069680 0x7fc3241644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 >> 172.21.15.118:0/1398422416 conn(0x7fc32406f420 msgr2=0x7fc324158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 shutdown_connections 2024-08-18T13:44:24.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:24.000+0000 7fc32a622700 1 -- 172.21.15.118:0/1398422416 wait complete. 2024-08-18T13:44:24.003 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 17 2024-08-18T13:44:24.013 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:44:26.014 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:26.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.124+0000 7f46b359e700 1 Processor -- start 2024-08-18T13:44:26.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.124+0000 7f46b359e700 1 -- start start 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b40c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f46b41561d0 0x7f46b4151f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b40c1110 con 0x7f46b40684e0 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40c0b90 con 0x7f46b41561d0 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40c8060 con 0x7f46b41556f0 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f46b41561d0 0x7f46b4151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f46b41561d0 0x7f46b4151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48914/0 (socket says 172.21.15.118:48914) 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 -- 172.21.15.118:0/3306778217 learned_addr learned my addr 172.21.15.118:0/3306778217 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b259c700 1 --2- 172.21.15.118:0/3306778217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b40c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:26.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 -- 172.21.15.118:0/3306778217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 msgr2=0x7f46b40c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 --2- 172.21.15.118:0/3306778217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b40c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 -- 172.21.15.118:0/3306778217 >> v1:172.21.15.178:6789/0 conn(0x7f46b40684e0 legacy=0x7f46b4155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 -- 172.21.15.118:0/3306778217 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40c8300 con 0x7f46b41561d0 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b2d9d700 1 --2- 172.21.15.118:0/3306778217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b41561d0 0x7f46b4151f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46a400d1d0 tx=0x7f46a400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fd8c7d282fecce server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b159a700 1 -- 172.21.15.118:0/3306778217 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4010070 con 0x7f46b41561d0 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b159a700 1 -- 172.21.15.118:0/3306778217 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a400de00 con 0x7f46b41561d0 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b159a700 1 -- 172.21.15.118:0/3306778217 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a400b320 con 0x7f46b41561d0 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- 172.21.15.118:0/3306778217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b41561d0 msgr2=0x7f46b4151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 --2- 172.21.15.118:0/3306778217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b41561d0 0x7f46b4151f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46a400d1d0 tx=0x7f46a400ba50 comp rx=0 tx=0).stop 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- 172.21.15.118:0/3306778217 shutdown_connections 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 --2- 172.21.15.118:0/3306778217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b41561d0 0x7f46b4151f50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 --2- 172.21.15.118:0/3306778217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b40c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- 172.21.15.118:0/3306778217 >> 172.21.15.118:0/3306778217 conn(0x7f46b406c020 msgr2=0x7f46b40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:26.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- 172.21.15.118:0/3306778217 shutdown_connections 2024-08-18T13:44:26.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.128+0000 7f46b359e700 1 -- 172.21.15.118:0/3306778217 wait complete. 2024-08-18T13:44:26.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 Processor -- start 2024-08-18T13:44:26.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 -- start start 2024-08-18T13:44:26.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b40684e0 0x7f46b4160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:26.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b4143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:26.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41561d0 0x7f46b413cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:26.136 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b414a360 con 0x7f46b40684e0 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4144b20 con 0x7f46b41561d0 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4144280 con 0x7f46b41556f0 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b40684e0 0x7f46b4160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b40684e0 0x7f46b4160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48926/0 (socket says 172.21.15.118:48926) 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 -- 172.21.15.118:0/1658166535 learned_addr learned my addr 172.21.15.118:0/1658166535 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b1d9b700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b4143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b2d9d700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41561d0 0x7f46b413cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 -- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 msgr2=0x7f46b4143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b4143950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 -- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41561d0 msgr2=0x7f46b413cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41561d0 0x7f46b413cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 -- 172.21.15.118:0/1658166535 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a400bcf0 con 0x7f46b40684e0 2024-08-18T13:44:26.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b1d9b700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b4143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:44:26.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b259c700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b40684e0 0x7f46b4160a60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00ec80 tx=0x7f46ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:26.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46aa7fc700 1 -- 172.21.15.118:0/1658166535 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00ca60 con 0x7f46b40684e0 2024-08-18T13:44:26.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46aa7fc700 1 -- 172.21.15.118:0/1658166535 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac00cbc0 con 0x7f46b40684e0 2024-08-18T13:44:26.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b4144560 con 0x7f46b40684e0 2024-08-18T13:44:26.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46aa7fc700 1 -- 172.21.15.118:0/1658166535 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac013470 con 0x7f46b40684e0 2024-08-18T13:44:26.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.132+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b4149ff0 con 0x7f46b40684e0 2024-08-18T13:44:26.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.136+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b40713a0 con 0x7f46b40684e0 2024-08-18T13:44:26.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.136+0000 7f46aa7fc700 1 -- 172.21.15.118:0/1658166535 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f46ac013820 con 0x7f46b40684e0 2024-08-18T13:44:26.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.136+0000 7f46aa7fc700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4694060860 0x7f4694062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:26.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.136+0000 7f46b1d9b700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4694060860 0x7f4694062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:26.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.136+0000 7f46aa7fc700 1 -- 172.21.15.118:0/1658166535 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f46ac07d440 con 0x7f46b40684e0 2024-08-18T13:44:26.140 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.136+0000 7f46b1d9b700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4694060860 0x7f4694062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f46b4068060 tx=0x7f469c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:26.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.144+0000 7f46aa7fc700 1 -- 172.21.15.118:0/1658166535 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f46ac04d430 con 0x7f46b40684e0 2024-08-18T13:44:26.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.344+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46b4145a70 con 0x7f46b40684e0 2024-08-18T13:44:26.348 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.344+0000 7f46aa7fc700 1 -- 172.21.15.118:0/1658166535 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5373 (secure 0 0 0) 0x7f46ac050a50 con 0x7f46b40684e0 2024-08-18T13:44:26.349 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:26.349 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:23.636905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:26.352 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4694060860 msgr2=0x7f4694062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:26.352 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4694060860 0x7f4694062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f46b4068060 tx=0x7f469c009040 comp rx=0 tx=0).stop 2024-08-18T13:44:26.352 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b40684e0 msgr2=0x7f46b4160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:26.352 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b40684e0 0x7f46b4160a60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00ec80 tx=0x7f46ac0091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.352 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 shutdown_connections 2024-08-18T13:44:26.353 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4694060860 0x7f4694062d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.353 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46b41561d0 0x7f46b413cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.353 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46b41556f0 0x7f46b4143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.353 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 --2- 172.21.15.118:0/1658166535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f46b40684e0 0x7f46b4160a60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:26.353 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 >> 172.21.15.118:0/1658166535 conn(0x7f46b406c020 msgr2=0x7f46b413c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:26.354 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 shutdown_connections 2024-08-18T13:44:26.354 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:26.348+0000 7f46b359e700 1 -- 172.21.15.118:0/1658166535 wait complete. 2024-08-18T13:44:26.354 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 17 2024-08-18T13:44:26.367 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:44:28.368 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:28.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.480+0000 7ff989900700 1 Processor -- start 2024-08-18T13:44:28.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.480+0000 7ff989900700 1 -- start start 2024-08-18T13:44:28.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.480+0000 7ff989900700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840aebc0 0x7ff9840b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:28.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.480+0000 7ff989900700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff9840aef90 0x7ff9840abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:28.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.480+0000 7ff989900700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff984155330 con 0x7ff9840c6880 2024-08-18T13:44:28.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.480+0000 7ff989900700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff984155140 con 0x7ff9840aef90 2024-08-18T13:44:28.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.480+0000 7ff989900700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9840af660 con 0x7ff9840aebc0 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff9840aef90 0x7ff9840abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff9840aef90 0x7ff9840abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48934/0 (socket says 172.21.15.118:48934) 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff9888fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840aebc0 0x7ff9840b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 -- 172.21.15.118:0/4219801700 learned_addr learned my addr 172.21.15.118:0/4219801700 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff9890ff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff9840c6880 0x7ff9840c6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37498/0 (socket says 172.21.15.118:37498) 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 -- 172.21.15.118:0/4219801700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840aebc0 msgr2=0x7ff9840b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 --2- 172.21.15.118:0/4219801700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840aebc0 0x7ff9840b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 -- 172.21.15.118:0/4219801700 >> v1:172.21.15.118:6789/0 conn(0x7ff9840c6880 legacy=0x7ff9840c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 -- 172.21.15.118:0/4219801700 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9841555d0 con 0x7ff9840aef90 2024-08-18T13:44:28.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff983fff700 1 --2- 172.21.15.118:0/4219801700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aef90 0x7ff9840abad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff9780099b0 tx=0x7ff978009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edf09e40e892adf server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff9837fe700 1 -- 172.21.15.118:0/4219801700 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978009ea0 con 0x7ff9840aef90 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff9837fe700 1 -- 172.21.15.118:0/4219801700 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800b860 con 0x7ff9840aef90 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff9837fe700 1 -- 172.21.15.118:0/4219801700 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800bc00 con 0x7ff9840aef90 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 -- 172.21.15.118:0/4219801700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aef90 msgr2=0x7ff9840abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 --2- 172.21.15.118:0/4219801700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aef90 0x7ff9840abad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff9780099b0 tx=0x7ff978009250 comp rx=0 tx=0).stop 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 -- 172.21.15.118:0/4219801700 shutdown_connections 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 --2- 172.21.15.118:0/4219801700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aef90 0x7ff9840abad0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 --2- 172.21.15.118:0/4219801700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840aebc0 0x7ff9840b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.488 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 -- 172.21.15.118:0/4219801700 >> 172.21.15.118:0/4219801700 conn(0x7ff9840cbba0 msgr2=0x7ff9840cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:28.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 -- 172.21.15.118:0/4219801700 shutdown_connections 2024-08-18T13:44:28.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.484+0000 7ff989900700 1 -- 172.21.15.118:0/4219801700 wait complete. 2024-08-18T13:44:28.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 Processor -- start 2024-08-18T13:44:28.489 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 -- start start 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aebc0 0x7ff984068270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9840aef90 0x7ff984069b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840c6880 0x7ff98406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98406a590 con 0x7ff9840aebc0 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9840c21f0 con 0x7ff9840aef90 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff984067de0 con 0x7ff9840c6880 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aebc0 0x7ff984068270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:28.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aebc0 0x7ff984068270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48936/0 (socket says 172.21.15.118:48936) 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 -- 172.21.15.118:0/3851660342 learned_addr learned my addr 172.21.15.118:0/3851660342 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9890ff700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840c6880 0x7ff98406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff983fff700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9840aef90 0x7ff984069b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 -- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840c6880 msgr2=0x7ff98406a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840c6880 0x7ff98406a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 -- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9840aef90 msgr2=0x7ff984069b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9840aef90 0x7ff984069b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 -- 172.21.15.118:0/3851660342 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9780094f0 con 0x7ff9840aebc0 2024-08-18T13:44:28.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff9888fe700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aebc0 0x7ff984068270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff97400ecb0 tx=0x7ff9740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:28.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff980ff9700 1 -- 172.21.15.118:0/3851660342 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97400c920 con 0x7ff9840aebc0 2024-08-18T13:44:28.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff980ff9700 1 -- 172.21.15.118:0/3851660342 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97400caa0 con 0x7ff9840aebc0 2024-08-18T13:44:28.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9840c2720 con 0x7ff9840aebc0 2024-08-18T13:44:28.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff980ff9700 1 -- 172.21.15.118:0/3851660342 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97400ce60 con 0x7ff9840aebc0 2024-08-18T13:44:28.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.488+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9840687b0 con 0x7ff9840aebc0 2024-08-18T13:44:28.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.492+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9840af920 con 0x7ff9840aebc0 2024-08-18T13:44:28.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.492+0000 7ff980ff9700 1 -- 172.21.15.118:0/3851660342 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff974013680 con 0x7ff9840aebc0 2024-08-18T13:44:28.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.492+0000 7ff980ff9700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff970060860 0x7ff970062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:28.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.492+0000 7ff980ff9700 1 -- 172.21.15.118:0/3851660342 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7ff97407d3a0 con 0x7ff9840aebc0 2024-08-18T13:44:28.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.492+0000 7ff983fff700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff970060860 0x7ff970062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:28.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.492+0000 7ff983fff700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff970060860 0x7ff970062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff9780099b0 tx=0x7ff978017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:28.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.496+0000 7ff980ff9700 1 -- 172.21.15.118:0/3851660342 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff9840af920 con 0x7ff9840aebc0 2024-08-18T13:44:28.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.696+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9840af920 con 0x7ff9840aebc0 2024-08-18T13:44:28.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.696+0000 7ff980ff9700 1 -- 172.21.15.118:0/3851660342 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5370 (secure 0 0 0) 0x7ff9840af920 con 0x7ff9840aebc0 2024-08-18T13:44:28.699 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:28.699 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:28.653598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:28.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff970060860 msgr2=0x7ff970062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:28.702 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff970060860 0x7ff970062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff9780099b0 tx=0x7ff978017000 comp rx=0 tx=0).stop 2024-08-18T13:44:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aebc0 msgr2=0x7ff984068270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aebc0 0x7ff984068270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff97400ecb0 tx=0x7ff9740091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 shutdown_connections 2024-08-18T13:44:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff970060860 0x7ff970062d10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9840c6880 0x7ff98406a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9840aef90 0x7ff984069b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 --2- 172.21.15.118:0/3851660342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9840aebc0 0x7ff984068270 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:28.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 >> 172.21.15.118:0/3851660342 conn(0x7ff9840cbba0 msgr2=0x7ff984157870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:28.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 shutdown_connections 2024-08-18T13:44:28.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:28.700+0000 7ff989900700 1 -- 172.21.15.118:0/3851660342 wait complete. 2024-08-18T13:44:28.705 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 18 2024-08-18T13:44:28.714 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 18, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_4805": { "addr": "172.21.15.178:6837/3452743394", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6836", "nonce": 3452743394, "type": "v2" }, { "addr": "172.21.15.178:6837", "nonce": 3452743394, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4805, "incarnation": 16, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:rejoin", "state_seq": 29 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:44:28.653598+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4805 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:44:30.719 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:30.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.832+0000 7f59cb158700 1 Processor -- start 2024-08-18T13:44:30.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.832+0000 7f59cb158700 1 -- start start 2024-08-18T13:44:30.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c40c8200 0x7f59c40673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:30.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c406f770 con 0x7f59c4153690 2024-08-18T13:44:30.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c406ee60 con 0x7f59c406f2f0 2024-08-18T13:44:30.837 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c415a640 con 0x7f59c40c8200 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59ca156700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f59c4153690 0x7f59c40c22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:37504/0 (socket says 172.21.15.118:37504) 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59ca156700 1 -- 172.21.15.118:0/2222335962 learned_addr learned my addr 172.21.15.118:0/2222335962 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9955700 1 --2- 172.21.15.118:0/2222335962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c40c8200 0x7f59c40673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138435956 0 0) 0x7f59b8001110 con 0x7f59c4153690 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b0003570 con 0x7f59c4153690 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078062325 0 0) 0x7f59c406ee60 con 0x7f59c406f2f0 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b8001110 con 0x7f59c406f2f0 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9955700 1 -- 172.21.15.118:0/2222335962 >> v1:172.21.15.178:6789/0 conn(0x7f59c406f2f0 legacy=0x7f59c40c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9955700 1 -- 172.21.15.118:0/2222335962 >> v1:172.21.15.118:6789/0 conn(0x7f59c4153690 legacy=0x7f59c40c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2025933665 0 0) 0x7f59b0003570 con 0x7f59c4153690 2024-08-18T13:44:30.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9955700 1 -- 172.21.15.118:0/2222335962 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4153d10 con 0x7f59c40c8200 2024-08-18T13:44:30.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9955700 1 --2- 172.21.15.118:0/2222335962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c40c8200 0x7f59c40673d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009910 tx=0x7f59c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6872a2625a3cc9b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:30.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0011630 con 0x7f59c40c8200 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0011790 con 0x7f59c40c8200 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59c9154700 1 -- 172.21.15.118:0/2222335962 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0011b30 con 0x7f59c40c8200 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- 172.21.15.118:0/2222335962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c40c8200 msgr2=0x7f59c40673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 --2- 172.21.15.118:0/2222335962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c40c8200 0x7f59c40673d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009910 tx=0x7f59c00091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- 172.21.15.118:0/2222335962 shutdown_connections 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 --2- 172.21.15.118:0/2222335962 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c40c8200 0x7f59c40673d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- 172.21.15.118:0/2222335962 >> 172.21.15.118:0/2222335962 conn(0x7f59c406c020 msgr2=0x7f59c40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- 172.21.15.118:0/2222335962 shutdown_connections 2024-08-18T13:44:30.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.836+0000 7f59cb158700 1 -- 172.21.15.118:0/2222335962 wait complete. 2024-08-18T13:44:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 Processor -- start 2024-08-18T13:44:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 -- start start 2024-08-18T13:44:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59c406f2f0 0x7f59c4156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c40c8200 0x7f59c40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c4153690 0x7f59c4155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40b4e70 con 0x7f59c406f2f0 2024-08-18T13:44:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c41543d0 con 0x7f59c40c8200 2024-08-18T13:44:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c4154990 con 0x7f59c4153690 2024-08-18T13:44:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59c406f2f0 0x7f59c4156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59c406f2f0 0x7f59c4156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48942/0 (socket says 172.21.15.118:48942) 2024-08-18T13:44:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 -- 172.21.15.118:0/2177858968 learned_addr learned my addr 172.21.15.118:0/2177858968 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca957700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c4153690 0x7f59c4155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59c9955700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c40c8200 0x7f59c40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 -- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c4153690 msgr2=0x7f59c4155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c4153690 0x7f59c4155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 -- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c40c8200 msgr2=0x7f59c40c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c40c8200 0x7f59c40c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 -- 172.21.15.118:0/2177858968 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0009420 con 0x7f59c406f2f0 2024-08-18T13:44:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59ca156700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59c406f2f0 0x7f59c4156c20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7e0 tx=0x7f59b8007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59b67fc700 1 -- 172.21.15.118:0/2177858968 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8007430 con 0x7f59c406f2f0 2024-08-18T13:44:30.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59b67fc700 1 -- 172.21.15.118:0/2177858968 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b80075b0 con 0x7f59c406f2f0 2024-08-18T13:44:30.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c4154e20 con 0x7f59c406f2f0 2024-08-18T13:44:30.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59b67fc700 1 -- 172.21.15.118:0/2177858968 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8004a80 con 0x7f59c406f2f0 2024-08-18T13:44:30.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.840+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c40b3fc0 con 0x7f59c406f2f0 2024-08-18T13:44:30.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.844+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c40b4300 con 0x7f59c406f2f0 2024-08-18T13:44:30.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.844+0000 7f59b67fc700 1 -- 172.21.15.118:0/2177858968 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f59b8014420 con 0x7f59c406f2f0 2024-08-18T13:44:30.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.844+0000 7f59b67fc700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59a8060890 0x7f59a8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:30.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.844+0000 7f59b67fc700 1 -- 172.21.15.118:0/2177858968 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f59b807ccf0 con 0x7f59c406f2f0 2024-08-18T13:44:30.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.844+0000 7f59c9955700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59a8060890 0x7f59a8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:30.848 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.844+0000 7f59c9955700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59a8060890 0x7f59a8062d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009860 tx=0x7f59c0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:30.853 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:30.852+0000 7f59b67fc700 1 -- 172.21.15.118:0/2177858968 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f59b804ccc0 con 0x7f59c406f2f0 2024-08-18T13:44:31.055 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.052+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59c406f770 con 0x7f59c406f2f0 2024-08-18T13:44:31.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.052+0000 7f59b67fc700 1 -- 172.21.15.118:0/2177858968 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f59c406f770 con 0x7f59c406f2f0 2024-08-18T13:44:31.056 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:31.056 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:28.653598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:31.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59a8060890 msgr2=0x7f59a8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:31.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59a8060890 0x7f59a8062d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009860 tx=0x7f59c0017000 comp rx=0 tx=0).stop 2024-08-18T13:44:31.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59c406f2f0 msgr2=0x7f59c4156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:31.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59c406f2f0 0x7f59c4156c20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7e0 tx=0x7f59b8007dd0 comp rx=0 tx=0).stop 2024-08-18T13:44:31.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 shutdown_connections 2024-08-18T13:44:31.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f59a8060890 0x7f59a8062d40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:31.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c4153690 0x7f59c4155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:31.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c40c8200 0x7f59c40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:31.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 --2- 172.21.15.118:0/2177858968 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59c406f2f0 0x7f59c4156c20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:31.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 >> 172.21.15.118:0/2177858968 conn(0x7f59c406c020 msgr2=0x7f59c40c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:31.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 shutdown_connections 2024-08-18T13:44:31.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:31.056+0000 7f59cb158700 1 -- 172.21.15.118:0/2177858968 wait complete. 2024-08-18T13:44:31.061 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 18 2024-08-18T13:44:31.070 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:44:33.071 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:33.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 Processor -- start 2024-08-18T13:44:33.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- start start 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec2406a890 0x7fec240c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fec241532e0 0x7fec240c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec240c0790 con 0x7fec240c7330 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240bf970 con 0x7fec2406a890 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240bfb80 con 0x7fec241532e0 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec2406a890 0x7fec240c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fec2406a890 0x7fec240c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48954/0 (socket says 172.21.15.118:48954) 2024-08-18T13:44:33.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 -- 172.21.15.118:0/2596870952 learned_addr learned my addr 172.21.15.118:0/2596870952 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:33.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec29b8a700 1 --2- 172.21.15.118:0/2596870952 >> v2:172.21.15.178:3300/0 conn(0x7fec241532e0 0x7fec240c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:33.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 -- 172.21.15.118:0/2596870952 >> v2:172.21.15.178:3300/0 conn(0x7fec241532e0 msgr2=0x7fec240c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:33.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 --2- 172.21.15.118:0/2596870952 >> v2:172.21.15.178:3300/0 conn(0x7fec241532e0 0x7fec240c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 -- 172.21.15.118:0/2596870952 >> v1:172.21.15.178:6789/0 conn(0x7fec240c7330 legacy=0x7fec2406ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:33.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 -- 172.21.15.118:0/2596870952 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240bfe60 con 0x7fec2406a890 2024-08-18T13:44:33.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec29b8a700 1 --2- 172.21.15.118:0/2596870952 >> v2:172.21.15.178:3300/0 conn(0x7fec241532e0 0x7fec240c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:33.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2a38b700 1 --2- 172.21.15.118:0/2596870952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec2406a890 0x7fec240c4600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec240c6600 tx=0x7fec18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f625a22068f2e53 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:33.189 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec29389700 1 -- 172.21.15.118:0/2596870952 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800b8f0 con 0x7fec2406a890 2024-08-18T13:44:33.189 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec29389700 1 -- 172.21.15.118:0/2596870952 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1800ba50 con 0x7fec2406a890 2024-08-18T13:44:33.189 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec29389700 1 -- 172.21.15.118:0/2596870952 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800bdf0 con 0x7fec2406a890 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2596870952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec2406a890 msgr2=0x7fec240c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2596870952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec2406a890 0x7fec240c4600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec240c6600 tx=0x7fec18009250 comp rx=0 tx=0).stop 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2596870952 shutdown_connections 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2596870952 >> v2:172.21.15.178:3300/0 conn(0x7fec241532e0 0x7fec240c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2596870952 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec2406a890 0x7fec240c4600 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2596870952 >> 172.21.15.118:0/2596870952 conn(0x7fec240c6700 msgr2=0x7fec240c6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2596870952 shutdown_connections 2024-08-18T13:44:33.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.184+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2596870952 wait complete. 2024-08-18T13:44:33.191 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 Processor -- start 2024-08-18T13:44:33.191 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 -- start start 2024-08-18T13:44:33.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec2406a890 0x7fec240a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:33.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec240c7330 0x7fec2406ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:33.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec241532e0 0x7fec24165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:33.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240a9e70 con 0x7fec241532e0 2024-08-18T13:44:33.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec24169aa0 con 0x7fec240c7330 2024-08-18T13:44:33.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2c5ef700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec240aa090 con 0x7fec2406a890 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec241532e0 0x7fec24165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec241532e0 0x7fec24165bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48968/0 (socket says 172.21.15.118:48968) 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 -- 172.21.15.118:0/2829564829 learned_addr learned my addr 172.21.15.118:0/2829564829 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec29b8a700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec240c7330 0x7fec2406ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2a38b700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec2406a890 0x7fec240a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 -- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec2406a890 msgr2=0x7fec240a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec2406a890 0x7fec240a9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 -- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec240c7330 msgr2=0x7fec2406ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec240c7330 0x7fec2406ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 -- 172.21.15.118:0/2829564829 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec180094f0 con 0x7fec241532e0 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec2ab8c700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec241532e0 0x7fec24165bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec1400c560 tx=0x7fec14007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.188+0000 7fec227fc700 1 -- 172.21.15.118:0/2829564829 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14011590 con 0x7fec241532e0 2024-08-18T13:44:33.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec227fc700 1 -- 172.21.15.118:0/2829564829 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec140116f0 con 0x7fec241532e0 2024-08-18T13:44:33.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec240aa370 con 0x7fec241532e0 2024-08-18T13:44:33.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec227fc700 1 -- 172.21.15.118:0/2829564829 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14011ab0 con 0x7fec241532e0 2024-08-18T13:44:33.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec24169450 con 0x7fec241532e0 2024-08-18T13:44:33.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec24169c50 con 0x7fec241532e0 2024-08-18T13:44:33.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec227fc700 1 -- 172.21.15.118:0/2829564829 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fec1401c440 con 0x7fec241532e0 2024-08-18T13:44:33.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec227fc700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fec10060860 0x7fec10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:33.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec227fc700 1 -- 172.21.15.118:0/2829564829 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fec1407c4c0 con 0x7fec241532e0 2024-08-18T13:44:33.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.192+0000 7fec2a38b700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fec10060860 0x7fec10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:33.197 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.196+0000 7fec2a38b700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fec10060860 0x7fec10062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec240c6600 tx=0x7fec18017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:33.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.204+0000 7fec227fc700 1 -- 172.21.15.118:0/2829564829 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fec1404c4b0 con 0x7fec241532e0 2024-08-18T13:44:33.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.400+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec240642f0 con 0x7fec241532e0 2024-08-18T13:44:33.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec227fc700 1 -- 172.21.15.118:0/2829564829 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fec14017090 con 0x7fec241532e0 2024-08-18T13:44:33.406 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:33.406 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:28.653598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:33.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fec10060860 msgr2=0x7fec10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:33.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fec10060860 0x7fec10062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec240c6600 tx=0x7fec18017000 comp rx=0 tx=0).stop 2024-08-18T13:44:33.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec241532e0 msgr2=0x7fec24165bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:33.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec241532e0 0x7fec24165bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec1400c560 tx=0x7fec14007ef0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 shutdown_connections 2024-08-18T13:44:33.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fec10060860 0x7fec10062d10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec241532e0 0x7fec24165bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec240c7330 0x7fec2406ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 --2- 172.21.15.118:0/2829564829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec2406a890 0x7fec240a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:33.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 >> 172.21.15.118:0/2829564829 conn(0x7fec240c6700 msgr2=0x7fec24152380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:33.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 shutdown_connections 2024-08-18T13:44:33.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:33.404+0000 7fec2c5ef700 1 -- 172.21.15.118:0/2829564829 wait complete. 2024-08-18T13:44:33.410 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 18 2024-08-18T13:44:33.420 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:44:35.420 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:35.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76b59e700 1 Processor -- start 2024-08-18T13:44:35.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76b59e700 1 -- start start 2024-08-18T13:44:35.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76b59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa76c14f410 0x7fa76c0ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa76c158830 0x7fa76c0abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76b59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c0c1c80 con 0x7fa76c14f7e0 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0c18e0 con 0x7fa76c158830 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76b59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0c09f0 con 0x7fa76c14f410 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa76c158830 0x7fa76c0abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa76c158830 0x7fa76c0abcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54292/0 (socket says 172.21.15.118:54292) 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 -- 172.21.15.118:0/942134217 learned_addr learned my addr 172.21.15.118:0/942134217 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:35.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76a59c700 1 --2- 172.21.15.118:0/942134217 >> v2:172.21.15.178:3300/0 conn(0x7fa76c14f410 0x7fa76c0ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 -- 172.21.15.118:0/942134217 >> v2:172.21.15.178:3300/0 conn(0x7fa76c14f410 msgr2=0x7fa76c0ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 --2- 172.21.15.118:0/942134217 >> v2:172.21.15.178:3300/0 conn(0x7fa76c14f410 0x7fa76c0ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 -- 172.21.15.118:0/942134217 >> v1:172.21.15.178:6789/0 conn(0x7fa76c14f7e0 legacy=0x7fa76c0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 -- 172.21.15.118:0/942134217 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c0c0cd0 con 0x7fa76c158830 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76ad9d700 1 --2- 172.21.15.118:0/942134217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0abcd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa76c088ac0 tx=0x7fa75c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b677f75a25d236 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.536+0000 7fa76959a700 1 -- 172.21.15.118:0/942134217 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c009e70 con 0x7fa76c158830 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76959a700 1 -- 172.21.15.118:0/942134217 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c00b8f0 con 0x7fa76c158830 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76959a700 1 -- 172.21.15.118:0/942134217 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00bc90 con 0x7fa76c158830 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- 172.21.15.118:0/942134217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 msgr2=0x7fa76c0abcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:35.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 --2- 172.21.15.118:0/942134217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0abcd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa76c088ac0 tx=0x7fa75c009250 comp rx=0 tx=0).stop 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- 172.21.15.118:0/942134217 shutdown_connections 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 --2- 172.21.15.118:0/942134217 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0abcd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 --2- 172.21.15.118:0/942134217 >> v2:172.21.15.178:3300/0 conn(0x7fa76c14f410 0x7fa76c0ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- 172.21.15.118:0/942134217 >> 172.21.15.118:0/942134217 conn(0x7fa76c0c8620 msgr2=0x7fa76c0c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- 172.21.15.118:0/942134217 shutdown_connections 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- 172.21.15.118:0/942134217 wait complete. 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 Processor -- start 2024-08-18T13:44:35.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- start start 2024-08-18T13:44:35.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c14f410 0x7fa76c157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:35.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c14f7e0 0x7fa76c0c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:35.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:35.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1661d0 con 0x7fa76c158830 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c165d00 con 0x7fa76c14f410 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76b59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c14a9d0 con 0x7fa76c14f7e0 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54304/0 (socket says 172.21.15.118:54304) 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76ad9d700 1 -- 172.21.15.118:0/2310777676 learned_addr learned my addr 172.21.15.118:0/2310777676 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa76a59c700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c14f410 0x7fa76c157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.540+0000 7fa769d9b700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c14f7e0 0x7fa76c0c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76ad9d700 1 -- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c14f7e0 msgr2=0x7fa76c0c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76ad9d700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c14f7e0 0x7fa76c0c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76ad9d700 1 -- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c14f410 msgr2=0x7fa76c157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76ad9d700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c14f410 0x7fa76c157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76ad9d700 1 -- 172.21.15.118:0/2310777676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0094f0 con 0x7fa76c158830 2024-08-18T13:44:35.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76ad9d700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0c7b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0896a0 tx=0x7fa75c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:35.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa75a7fc700 1 -- 172.21.15.118:0/2310777676 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00be40 con 0x7fa76c158830 2024-08-18T13:44:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa75a7fc700 1 -- 172.21.15.118:0/2310777676 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c01a630 con 0x7fa76c158830 2024-08-18T13:44:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c165f30 con 0x7fa76c158830 2024-08-18T13:44:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa75a7fc700 1 -- 172.21.15.118:0/2310777676 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c01a9f0 con 0x7fa76c158830 2024-08-18T13:44:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c14a0e0 con 0x7fa76c158830 2024-08-18T13:44:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c0713a0 con 0x7fa76c158830 2024-08-18T13:44:35.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.544+0000 7fa75a7fc700 1 -- 172.21.15.118:0/2310777676 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa75c0253f0 con 0x7fa76c158830 2024-08-18T13:44:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.548+0000 7fa75a7fc700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa74c060860 0x7fa74c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.548+0000 7fa75a7fc700 1 -- 172.21.15.118:0/2310777676 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fa75c07ea40 con 0x7fa76c158830 2024-08-18T13:44:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.548+0000 7fa76a59c700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa74c060860 0x7fa74c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.548+0000 7fa76a59c700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa74c060860 0x7fa74c062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa76400f570 tx=0x7fa76400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:35.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.552+0000 7fa75a7fc700 1 -- 172.21.15.118:0/2310777676 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa75c04e990 con 0x7fa76c158830 2024-08-18T13:44:35.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.760+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa76c14ab10 con 0x7fa76c158830 2024-08-18T13:44:35.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.760+0000 7fa75a7fc700 1 -- 172.21.15.118:0/2310777676 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fa75c051fb0 con 0x7fa76c158830 2024-08-18T13:44:35.763 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:35.763 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:35.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa74c060860 msgr2=0x7fa74c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:35.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa74c060860 0x7fa74c062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa76400f570 tx=0x7fa76400b040 comp rx=0 tx=0).stop 2024-08-18T13:44:35.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 msgr2=0x7fa76c0c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:35.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0c7b30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0896a0 tx=0x7fa75c017000 comp rx=0 tx=0).stop 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 shutdown_connections 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa74c060860 0x7fa74c062d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa76c158830 0x7fa76c0c7b30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c14f7e0 0x7fa76c0c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 --2- 172.21.15.118:0/2310777676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c14f410 0x7fa76c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 >> 172.21.15.118:0/2310777676 conn(0x7fa76c0c8620 msgr2=0x7fa76c0ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 shutdown_connections 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:35.764+0000 7fa76b59e700 1 -- 172.21.15.118:0/2310777676 wait complete. 2024-08-18T13:44:35.766 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 19 2024-08-18T13:44:35.776 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=0, replacing gid=4306 2024-08-18T13:44:35.776 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.f 2024-08-18T13:44:40.711 INFO:tasks.ceph.mds.d.smithi178.stderr:2024-08-18T13:44:40.705+0000 7f5142816700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T13:44:40.777 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-08-18T13:44:40.778 INFO:tasks.ceph.mds.f:Restarting daemon 2024-08-18T13:44:40.778 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-08-18T13:44:40.781 INFO:tasks.ceph.mds.f:Started 2024-08-18T13:44:40.781 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:40.859 INFO:tasks.ceph.mds.f.smithi178.stdout:starting mds.f at 2024-08-18T13:44:40.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5d46c700 1 Processor -- start 2024-08-18T13:44:40.895 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5d46c700 1 -- start start 2024-08-18T13:44:40.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5d46c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0c580aebc0 0x7f0c580b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:40.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5d46c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c58155330 con 0x7f0c580c6880 2024-08-18T13:44:40.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5d46c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c581555d0 con 0x7f0c580aef90 2024-08-18T13:44:40.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5d46c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c58155140 con 0x7f0c580aebc0 2024-08-18T13:44:40.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5cc6b700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f0c580c6880 0x7f0c580c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55200/0 (socket says 172.21.15.118:55200) 2024-08-18T13:44:40.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c57fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0c580aebc0 0x7f0c580b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:40.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c5cc6b700 1 -- 172.21.15.118:0/1750957388 learned_addr learned my addr 172.21.15.118:0/1750957388 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:40.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c57fff700 1 -- 172.21.15.118:0/1750957388 >> v1:172.21.15.178:6789/0 conn(0x7f0c580aef90 legacy=0x7f0c580abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:40.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c57fff700 1 -- 172.21.15.118:0/1750957388 >> v1:172.21.15.118:6789/0 conn(0x7f0c580c6880 legacy=0x7f0c580c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:40.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.892+0000 7f0c57fff700 1 -- 172.21.15.118:0/1750957388 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58155870 con 0x7f0c580aebc0 2024-08-18T13:44:40.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c57fff700 1 --2- 172.21.15.118:0/1750957388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580aebc0 0x7f0c580b1640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0c58155a60 tx=0x7f0c480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613c54b2fac97f14 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:40.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c577fe700 1 -- 172.21.15.118:0/1750957388 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800b6b0 con 0x7f0c580aebc0 2024-08-18T13:44:40.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c577fe700 1 -- 172.21.15.118:0/1750957388 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4800b810 con 0x7f0c580aebc0 2024-08-18T13:44:40.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c577fe700 1 -- 172.21.15.118:0/1750957388 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800bbd0 con 0x7f0c580aebc0 2024-08-18T13:44:40.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c5d46c700 1 -- 172.21.15.118:0/1750957388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580aebc0 msgr2=0x7f0c580b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:40.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/1750957388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580aebc0 0x7f0c580b1640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0c58155a60 tx=0x7f0c480091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:40.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c5d46c700 1 -- 172.21.15.118:0/1750957388 shutdown_connections 2024-08-18T13:44:40.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/1750957388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580aebc0 0x7f0c580b1640 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:40.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c5d46c700 1 -- 172.21.15.118:0/1750957388 >> 172.21.15.118:0/1750957388 conn(0x7f0c580cbba0 msgr2=0x7f0c580cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:40.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c5d46c700 1 -- 172.21.15.118:0/1750957388 shutdown_connections 2024-08-18T13:44:40.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.896+0000 7f0c5d46c700 1 -- 172.21.15.118:0/1750957388 wait complete. 2024-08-18T13:44:40.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 Processor -- start 2024-08-18T13:44:40.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 -- start start 2024-08-18T13:44:40.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c580aebc0 0x7f0c58069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:40.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c580aef90 0x7f0c580ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:40.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580c6880 0x7f0c580c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:40.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c58154850 con 0x7f0c580c6880 2024-08-18T13:44:40.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c58069b00 con 0x7f0c580aef90 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c58154a80 con 0x7f0c580aebc0 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580c6880 0x7f0c580c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580c6880 0x7f0c580c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54330/0 (socket says 172.21.15.118:54330) 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 -- 172.21.15.118:0/3717378587 learned_addr learned my addr 172.21.15.118:0/3717378587 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c4ffff700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c580aef90 0x7f0c580ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c57fff700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c580aebc0 0x7f0c58069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 -- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c580aebc0 msgr2=0x7f0c58069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c580aebc0 0x7f0c58069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 -- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c580aef90 msgr2=0x7f0c580ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c580aef90 0x7f0c580ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:40.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 -- 172.21.15.118:0/3717378587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48009420 con 0x7f0c580c6880 2024-08-18T13:44:40.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c57fff700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c580aebc0 0x7f0c58069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:40.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5cc6b700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580c6880 0x7f0c580c2270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000cbf0 tx=0x7f0c500019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:40.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c54ff9700 1 -- 172.21.15.118:0/3717378587 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000a8d0 con 0x7f0c580c6880 2024-08-18T13:44:40.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c54ff9700 1 -- 172.21.15.118:0/3717378587 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5000aa30 con 0x7f0c580c6880 2024-08-18T13:44:40.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c58154d60 con 0x7f0c580c6880 2024-08-18T13:44:40.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c54ff9700 1 -- 172.21.15.118:0/3717378587 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000adb0 con 0x7f0c580c6880 2024-08-18T13:44:40.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.900+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c580bf980 con 0x7f0c580c6880 2024-08-18T13:44:40.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.904+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c58069cd0 con 0x7f0c580c6880 2024-08-18T13:44:40.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.904+0000 7f0c54ff9700 1 -- 172.21.15.118:0/3717378587 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0c50007300 con 0x7f0c580c6880 2024-08-18T13:44:40.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.904+0000 7f0c54ff9700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0c38060860 0x7f0c38062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:40.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.904+0000 7f0c57fff700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0c38060860 0x7f0c38062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:40.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.904+0000 7f0c54ff9700 1 -- 172.21.15.118:0/3717378587 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f0c5007d2c0 con 0x7f0c580c6880 2024-08-18T13:44:40.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.904+0000 7f0c57fff700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0c38060860 0x7f0c38062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c480091e0 tx=0x7f0c48016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:40.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:40.908+0000 7f0c54ff9700 1 -- 172.21.15.118:0/3717378587 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0c58069cd0 con 0x7f0c580c6880 2024-08-18T13:44:41.119 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.116+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c58069cd0 con 0x7f0c580c6880 2024-08-18T13:44:41.119 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.116+0000 7f0c54ff9700 1 -- 172.21.15.118:0/3717378587 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f0c58069cd0 con 0x7f0c580c6880 2024-08-18T13:44:41.120 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:41.120 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0c38060860 msgr2=0x7f0c38062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0c38060860 0x7f0c38062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c480091e0 tx=0x7f0c48016000 comp rx=0 tx=0).stop 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580c6880 msgr2=0x7f0c580c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580c6880 0x7f0c580c2270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000cbf0 tx=0x7f0c500019c0 comp rx=0 tx=0).stop 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 shutdown_connections 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0c38060860 0x7f0c38062d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c580c6880 0x7f0c580c2270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:41.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c580aef90 0x7f0c580ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:41.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 --2- 172.21.15.118:0/3717378587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c580aebc0 0x7f0c58069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:41.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 >> 172.21.15.118:0/3717378587 conn(0x7f0c580cbba0 msgr2=0x7f0c580cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:41.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 shutdown_connections 2024-08-18T13:44:41.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:41.120+0000 7f0c5d46c700 1 -- 172.21.15.118:0/3717378587 wait complete. 2024-08-18T13:44:41.124 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 19 2024-08-18T13:44:41.139 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-08-18T13:44:41.478 INFO:tasks.workunit.client.0.smithi118.stderr:error: RPC failed; curl 56 GnuTLS recv error (-110): The TLS connection was non-properly terminated. 2024-08-18T13:44:41.489 INFO:tasks.workunit.client.0.smithi118.stderr:fatal: the remote end hung up unexpectedly 2024-08-18T13:44:41.505 INFO:tasks.workunit.client.0.smithi118.stderr:fatal: early EOF 2024-08-18T13:44:41.506 INFO:tasks.workunit.client.0.smithi118.stderr:fatal: index-pack failed 2024-08-18T13:44:41.834 INFO:tasks.workunit.client.0.smithi118.stderr:+ retry 2024-08-18T13:44:42.000 INFO:tasks.workunit.client.0.smithi118.stderr:+ rm -rf ceph 2024-08-18T13:44:42.001 INFO:tasks.workunit.client.0.smithi118.stderr:+ timeout 3600 git clone https://git.ceph.com/ceph.git 2024-08-18T13:44:42.001 INFO:tasks.workunit.client.0.smithi118.stderr:Cloning into 'ceph'... 2024-08-18T13:44:43.007 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:44:43.009 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:44:43.140 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:44:43.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd9e0e700 1 Processor -- start 2024-08-18T13:44:43.262 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd9e0e700 1 -- start start 2024-08-18T13:44:43.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd9e0e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd414f410 0x7f7fd40ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:43.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd9e0e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7fd4158830 0x7f7fd40abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:43.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd9e0e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd40c1c80 con 0x7f7fd414f7e0 2024-08-18T13:44:43.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd9e0e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd40c18e0 con 0x7f7fd4158830 2024-08-18T13:44:43.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd9e0e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd40c09f0 con 0x7f7fd414f410 2024-08-18T13:44:43.264 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd3fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f7fd414f7e0 0x7f7fd40c7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55214/0 (socket says 172.21.15.118:55214) 2024-08-18T13:44:43.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd3fff700 1 -- 172.21.15.118:0/2500271940 learned_addr learned my addr 172.21.15.118:0/2500271940 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:44:43.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd8e0c700 1 --2- 172.21.15.118:0/2500271940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd414f410 0x7f7fd40ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:43.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd960d700 1 --2- 172.21.15.118:0/2500271940 >> v2:172.21.15.178:3300/0 conn(0x7f7fd4158830 0x7f7fd40abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:43.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd37fe700 1 -- 172.21.15.118:0/2500271940 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516626938 0 0) 0x7f7fd40c1c80 con 0x7f7fd414f7e0 2024-08-18T13:44:43.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd37fe700 1 -- 172.21.15.118:0/2500271940 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc4003570 con 0x7f7fd414f7e0 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd37fe700 1 -- 172.21.15.118:0/2500271940 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916341877 0 0) 0x7f7fc4003570 con 0x7f7fd414f7e0 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd8e0c700 1 -- 172.21.15.118:0/2500271940 >> v2:172.21.15.178:3300/0 conn(0x7f7fd4158830 msgr2=0x7f7fd40abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd8e0c700 1 --2- 172.21.15.118:0/2500271940 >> v2:172.21.15.178:3300/0 conn(0x7f7fd4158830 0x7f7fd40abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd8e0c700 1 -- 172.21.15.118:0/2500271940 >> v1:172.21.15.118:6789/0 conn(0x7f7fd414f7e0 legacy=0x7f7fd40c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd8e0c700 1 -- 172.21.15.118:0/2500271940 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd40c0cd0 con 0x7f7fd414f410 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.260+0000 7f7fd960d700 1 --2- 172.21.15.118:0/2500271940 >> v2:172.21.15.178:3300/0 conn(0x7f7fd4158830 0x7f7fd40abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd8e0c700 1 --2- 172.21.15.118:0/2500271940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd414f410 0x7f7fd40ab790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009910 tx=0x7f7fc80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b6a36a7a6dd91 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd37fe700 1 -- 172.21.15.118:0/2500271940 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0001470 con 0x7f7fd414f410 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd37fe700 1 -- 172.21.15.118:0/2500271940 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc8011420 con 0x7f7fd414f410 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd37fe700 1 -- 172.21.15.118:0/2500271940 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80117a0 con 0x7f7fd414f410 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/2500271940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd414f410 msgr2=0x7f7fd40ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:43.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/2500271940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd414f410 0x7f7fd40ab790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009910 tx=0x7f7fc80091a0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/2500271940 shutdown_connections 2024-08-18T13:44:43.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/2500271940 >> v2:172.21.15.178:3300/0 conn(0x7f7fd4158830 0x7f7fd40abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/2500271940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd414f410 0x7f7fd40ab790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/2500271940 >> 172.21.15.118:0/2500271940 conn(0x7f7fd40c8620 msgr2=0x7f7fd40c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:43.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/2500271940 shutdown_connections 2024-08-18T13:44:43.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/2500271940 wait complete. 2024-08-18T13:44:43.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 Processor -- start 2024-08-18T13:44:43.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- start start 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fd414f410 0x7f7fd40c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fd414f7e0 0x7f7fd414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd4158830 0x7f7fd414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4067b30 con 0x7f7fd414f7e0 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd414b770 con 0x7f7fd414f410 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd9e0e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4165290 con 0x7f7fd4158830 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fd414f7e0 0x7f7fd414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fd414f7e0 0x7f7fd414c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54334/0 (socket says 172.21.15.118:54334) 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd3fff700 1 -- 172.21.15.118:0/1124410103 learned_addr learned my addr 172.21.15.118:0/1124410103 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.264+0000 7f7fd8e0c700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fd414f410 0x7f7fd40c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd960d700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd4158830 0x7f7fd414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd3fff700 1 -- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd4158830 msgr2=0x7f7fd414c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:43.271 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd3fff700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd4158830 0x7f7fd414c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd3fff700 1 -- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fd414f410 msgr2=0x7f7fd40c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd3fff700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fd414f410 0x7f7fd40c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd3fff700 1 -- 172.21.15.118:0/1124410103 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc009a20 con 0x7f7fd414f7e0 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd8e0c700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fd414f410 0x7f7fd40c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd3fff700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fd414f7e0 0x7f7fd414c210 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000cb90 tx=0x7f7fc0001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd0ff9700 1 -- 172.21.15.118:0/1124410103 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0007360 con 0x7f7fd414f7e0 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd0ff9700 1 -- 172.21.15.118:0/1124410103 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc00074e0 con 0x7f7fd414f7e0 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc8009420 con 0x7f7fd414f7e0 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd40c6690 con 0x7f7fd414f7e0 2024-08-18T13:44:43.272 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd0ff9700 1 -- 172.21.15.118:0/1124410103 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0004830 con 0x7f7fd414f7e0 2024-08-18T13:44:43.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd40713a0 con 0x7f7fd414f7e0 2024-08-18T13:44:43.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.268+0000 7f7fd0ff9700 1 -- 172.21.15.118:0/1124410103 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7fc0004be0 con 0x7f7fd414f7e0 2024-08-18T13:44:43.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.272+0000 7f7fd0ff9700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7fb8060860 0x7f7fb8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:44:43.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.272+0000 7f7fd0ff9700 1 -- 172.21.15.118:0/1124410103 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f7fc007c950 con 0x7f7fd414f7e0 2024-08-18T13:44:43.273 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.272+0000 7f7fd8e0c700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7fb8060860 0x7f7fb8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:44:43.274 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.272+0000 7f7fd8e0c700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7fb8060860 0x7f7fb8062d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80098e0 tx=0x7f7fc8016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:44:43.279 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.276+0000 7f7fd0ff9700 1 -- 172.21.15.118:0/1124410103 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7fc004c940 con 0x7f7fd414f7e0 2024-08-18T13:44:43.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.488+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fd405e660 con 0x7f7fd414f7e0 2024-08-18T13:44:43.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.488+0000 7f7fd0ff9700 1 -- 172.21.15.118:0/1124410103 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f7fc0018020 con 0x7f7fd414f7e0 2024-08-18T13:44:43.491 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:44:43.491 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:44:43.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7fb8060860 msgr2=0x7f7fb8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:43.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7fb8060860 0x7f7fb8062d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80098e0 tx=0x7f7fc8016000 comp rx=0 tx=0).stop 2024-08-18T13:44:43.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fd414f7e0 msgr2=0x7f7fd414c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:44:43.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fd414f7e0 0x7f7fd414c210 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000cb90 tx=0x7f7fc0001b50 comp rx=0 tx=0).stop 2024-08-18T13:44:43.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 shutdown_connections 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7fb8060860 0x7f7fb8062d10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fd4158830 0x7f7fd414c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fd414f7e0 0x7f7fd414c210 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 --2- 172.21.15.118:0/1124410103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fd414f410 0x7f7fd40c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 >> 172.21.15.118:0/1124410103 conn(0x7f7fd40c8620 msgr2=0x7f7fd4157600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 shutdown_connections 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:44:43.492+0000 7f7fd9e0e700 1 -- 172.21.15.118:0/1124410103 wait complete. 2024-08-18T13:44:43.494 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:44:43.505 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-08-18T13:44:43.505 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 35 secs before thrashing 2024-08-18T13:45:13.059 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:45:13.063 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:45:18.506 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:18.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e7d04e700 1 Processor -- start 2024-08-18T13:45:18.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e7d04e700 1 -- start start 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e7d04e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1e780aebc0 0x7f1e780b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e7d04e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e780abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e7d04e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e78155330 con 0x7f1e780c6880 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e7d04e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e78155140 con 0x7f1e780aebc0 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e7d04e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780af660 con 0x7f1e780aef90 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e780abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e77fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1e780aebc0 0x7f1e780b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:18.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e780abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:36524/0 (socket says 172.21.15.118:36524) 2024-08-18T13:45:18.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 -- 172.21.15.118:0/1369031580 learned_addr learned my addr 172.21.15.118:0/1369031580 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:18.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 -- 172.21.15.118:0/1369031580 >> v2:172.21.15.178:3300/0 conn(0x7f1e780aebc0 msgr2=0x7f1e780b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:18.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 --2- 172.21.15.118:0/1369031580 >> v2:172.21.15.178:3300/0 conn(0x7f1e780aebc0 0x7f1e780b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 -- 172.21.15.118:0/1369031580 >> v1:172.21.15.178:6789/0 conn(0x7f1e780c6880 legacy=0x7f1e780c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:18.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 -- 172.21.15.118:0/1369031580 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e781555d0 con 0x7f1e780aef90 2024-08-18T13:45:18.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e6ffff700 1 --2- 172.21.15.118:0/1369031580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e780abad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e600099b0 tx=0x7f1e60009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93aeedb934390727 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:18.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e777fe700 1 -- 172.21.15.118:0/1369031580 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60009ea0 con 0x7f1e780aef90 2024-08-18T13:45:18.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e777fe700 1 -- 172.21.15.118:0/1369031580 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6000b860 con 0x7f1e780aef90 2024-08-18T13:45:18.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.627+0000 7f1e777fe700 1 -- 172.21.15.118:0/1369031580 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000bc00 con 0x7f1e780aef90 2024-08-18T13:45:18.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 -- 172.21.15.118:0/1369031580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 msgr2=0x7f1e780abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:18.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/1369031580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e780abad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e600099b0 tx=0x7f1e60009250 comp rx=0 tx=0).stop 2024-08-18T13:45:18.635 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 -- 172.21.15.118:0/1369031580 shutdown_connections 2024-08-18T13:45:18.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/1369031580 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e780abad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/1369031580 >> v2:172.21.15.178:3300/0 conn(0x7f1e780aebc0 0x7f1e780b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 -- 172.21.15.118:0/1369031580 >> 172.21.15.118:0/1369031580 conn(0x7f1e780cbba0 msgr2=0x7f1e780cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:18.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 -- 172.21.15.118:0/1369031580 shutdown_connections 2024-08-18T13:45:18.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 -- 172.21.15.118:0/1369031580 wait complete. 2024-08-18T13:45:18.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 Processor -- start 2024-08-18T13:45:18.636 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.631+0000 7f1e7d04e700 1 -- start start 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e780aebc0 0x7f1e780a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e7814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e780c6880 0x7f1e780aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78168590 con 0x7f1e780aebc0 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780aaef0 con 0x7f1e780c6880 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780a7790 con 0x7f1e780aef90 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e780aebc0 0x7f1e780a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:18.639 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e780aebc0 0x7f1e780a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:46716/0 (socket says 172.21.15.118:46716) 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 -- 172.21.15.118:0/3414208594 learned_addr learned my addr 172.21.15.118:0/3414208594 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e6ffff700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e7814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7c84d700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e780c6880 0x7f1e780aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 -- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 msgr2=0x7f1e7814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e7814e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 -- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e780c6880 msgr2=0x7f1e780aa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e780c6880 0x7f1e780aa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 -- 172.21.15.118:0/3414208594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e600094f0 con 0x7f1e780aebc0 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e77fff700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e780aebc0 0x7f1e780a7170 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800ec80 tx=0x7f1e680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e74ff9700 1 -- 172.21.15.118:0/3414208594 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800c850 con 0x7f1e780aebc0 2024-08-18T13:45:18.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e74ff9700 1 -- 172.21.15.118:0/3414208594 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800c9b0 con 0x7f1e780aebc0 2024-08-18T13:45:18.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e781687e0 con 0x7f1e780aebc0 2024-08-18T13:45:18.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e74ff9700 1 -- 172.21.15.118:0/3414208594 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800cd50 con 0x7f1e780aebc0 2024-08-18T13:45:18.641 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e78168da0 con 0x7f1e780aebc0 2024-08-18T13:45:18.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.635+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e780ab0b0 con 0x7f1e780aebc0 2024-08-18T13:45:18.642 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.639+0000 7f1e74ff9700 1 -- 172.21.15.118:0/3414208594 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1e68013680 con 0x7f1e780aebc0 2024-08-18T13:45:18.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.639+0000 7f1e74ff9700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1e58060860 0x7f1e58062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:18.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.639+0000 7f1e74ff9700 1 -- 172.21.15.118:0/3414208594 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f1e6807d230 con 0x7f1e780aebc0 2024-08-18T13:45:18.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.639+0000 7f1e6ffff700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1e58060860 0x7f1e58062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:18.643 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.639+0000 7f1e6ffff700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1e58060860 0x7f1e58062d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b1b80 tx=0x7f1e60017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:18.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.643+0000 7f1e74ff9700 1 -- 172.21.15.118:0/3414208594 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1e6804d220 con 0x7f1e780aebc0 2024-08-18T13:45:18.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.871+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e780713a0 con 0x7f1e780aebc0 2024-08-18T13:45:18.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.871+0000 7f1e74ff9700 1 -- 172.21.15.118:0/3414208594 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f1e68050840 con 0x7f1e780aebc0 2024-08-18T13:45:18.876 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:18.876 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:18.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1e58060860 msgr2=0x7f1e58062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1e58060860 0x7f1e58062d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1e780b1b80 tx=0x7f1e60017000 comp rx=0 tx=0).stop 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e780aebc0 msgr2=0x7f1e780a7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e780aebc0 0x7f1e780a7170 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800ec80 tx=0x7f1e680091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 shutdown_connections 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1e58060860 0x7f1e58062d10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e780c6880 0x7f1e780aa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e780aef90 0x7f1e7814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 --2- 172.21.15.118:0/3414208594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e780aebc0 0x7f1e780a7170 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800ec80 tx=0x7f1e680091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:18.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 >> 172.21.15.118:0/3414208594 conn(0x7f1e780cbba0 msgr2=0x7f1e781576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:18.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 shutdown_connections 2024-08-18T13:45:18.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:18.875+0000 7f1e7d04e700 1 -- 172.21.15.118:0/3414208594 wait complete. 2024-08-18T13:45:18.880 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:18.890 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=0) 2024-08-18T13:45:18.891 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-18T13:45:18.891 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:45:18.909 INFO:tasks.ceph.mds.d:Stopped 2024-08-18T13:45:18.910 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:19.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.015+0000 7fb80739d700 1 Processor -- start 2024-08-18T13:45:19.020 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.015+0000 7fb80739d700 1 -- start start 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb8000c0cf0 0x7fb800069d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb800068d50 con 0x7fb80014f840 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8000698b0 con 0x7fb800068260 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8000c2270 con 0x7fb8000c0cf0 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80593a700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fb80014f840 0x7fb80015fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:44124/0 (socket says 172.21.15.118:44124) 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80593a700 1 -- 172.21.15.118:0/340137567 learned_addr learned my addr 172.21.15.118:0/340137567 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb805139700 1 --2- 172.21.15.118:0/340137567 >> v2:172.21.15.178:3300/0 conn(0x7fb8000c0cf0 0x7fb800069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239627752 0 0) 0x7fb800068d50 con 0x7fb80014f840 2024-08-18T13:45:19.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7ec003570 con 0x7fb80014f840 2024-08-18T13:45:19.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008614468 0 0) 0x7fb8000698b0 con 0x7fb800068260 2024-08-18T13:45:19.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb800068d50 con 0x7fb800068260 2024-08-18T13:45:19.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb805139700 1 -- 172.21.15.118:0/340137567 >> v1:172.21.15.178:6789/0 conn(0x7fb800068260 legacy=0x7fb80014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:19.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb805139700 1 -- 172.21.15.118:0/340137567 >> v1:172.21.15.118:6789/0 conn(0x7fb80014f840 legacy=0x7fb80015fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:19.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3417883101 0 0) 0x7fb7ec003570 con 0x7fb80014f840 2024-08-18T13:45:19.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb805139700 1 -- 172.21.15.118:0/340137567 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb800156090 con 0x7fb8000c0cf0 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb805139700 1 --2- 172.21.15.118:0/340137567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800069d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0098e0 tx=0x7fb7fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63acb1ee2ef9a5b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80017f0 con 0x7fb8000c0cf0 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc011420 con 0x7fb8000c0cf0 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb7f7fff700 1 -- 172.21.15.118:0/340137567 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0117c0 con 0x7fb8000c0cf0 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- 172.21.15.118:0/340137567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 msgr2=0x7fb800069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 --2- 172.21.15.118:0/340137567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800069d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0098e0 tx=0x7fb7fc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- 172.21.15.118:0/340137567 shutdown_connections 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 --2- 172.21.15.118:0/340137567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800069d40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- 172.21.15.118:0/340137567 >> 172.21.15.118:0/340137567 conn(0x7fb800067e60 msgr2=0x7fb8000cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- 172.21.15.118:0/340137567 shutdown_connections 2024-08-18T13:45:19.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.019+0000 7fb80739d700 1 -- 172.21.15.118:0/340137567 wait complete. 2024-08-18T13:45:19.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 Processor -- start 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 -- start start 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb800068260 0x7fb800168200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb80014f840 0x7fb800143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb800150a80 con 0x7fb80014f840 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb800152c60 con 0x7fb8000c0cf0 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb800143760 con 0x7fb800068260 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb80014f840 0x7fb800143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb80014f840 0x7fb800143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:46730/0 (socket says 172.21.15.118:46730) 2024-08-18T13:45:19.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 -- 172.21.15.118:0/2990283069 learned_addr learned my addr 172.21.15.118:0/2990283069 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb805139700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb800068260 0x7fb800168200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb804938700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 -- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb800068260 msgr2=0x7fb800168200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb800068260 0x7fb800168200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 -- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 msgr2=0x7fb800150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 -- 172.21.15.118:0/2990283069 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc009420 con 0x7fb80014f840 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb804938700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:45:19.032 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb805139700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb800068260 0x7fb800168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80593a700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb80014f840 0x7fb800143220 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800ccc0 tx=0x7fb7f8007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb7f57fa700 1 -- 172.21.15.118:0/2990283069 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800ce70 con 0x7fb80014f840 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb7f57fa700 1 -- 172.21.15.118:0/2990283069 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f8004dc0 con 0x7fb80014f840 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8001523c0 con 0x7fb80014f840 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb7f57fa700 1 -- 172.21.15.118:0/2990283069 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800ab50 con 0x7fb80014f840 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.023+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb800152980 con 0x7fb80014f840 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.027+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8000642f0 con 0x7fb80014f840 2024-08-18T13:45:19.033 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.027+0000 7fb7f57fa700 1 -- 172.21.15.118:0/2990283069 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb7f8007150 con 0x7fb80014f840 2024-08-18T13:45:19.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.027+0000 7fb7f57fa700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e4060860 0x7fb7e4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:19.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.027+0000 7fb805139700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e4060860 0x7fb7e4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:19.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.027+0000 7fb7f57fa700 1 -- 172.21.15.118:0/2990283069 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fb7f807ca50 con 0x7fb80014f840 2024-08-18T13:45:19.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.027+0000 7fb805139700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e4060860 0x7fb7e4062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0093f0 tx=0x7fb7fc016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:19.036 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.031+0000 7fb7f57fa700 1 -- 172.21.15.118:0/2990283069 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb7f804cac0 con 0x7fb80014f840 2024-08-18T13:45:19.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.239+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8000c6a50 con 0x7fb80014f840 2024-08-18T13:45:19.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.239+0000 7fb7f57fa700 1 -- 172.21.15.118:0/2990283069 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7fb7f8018090 con 0x7fb80014f840 2024-08-18T13:45:19.255 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:19.255 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:19.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e4060860 msgr2=0x7fb7e4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:19.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e4060860 0x7fb7e4062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0093f0 tx=0x7fb7fc016000 comp rx=0 tx=0).stop 2024-08-18T13:45:19.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb80014f840 msgr2=0x7fb800143220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:19.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb80014f840 0x7fb800143220 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800ccc0 tx=0x7fb7f8007d20 comp rx=0 tx=0).stop 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 shutdown_connections 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb7e4060860 0x7fb7e4062d10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb80014f840 0x7fb800143220 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8000c0cf0 0x7fb800150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 --2- 172.21.15.118:0/2990283069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb800068260 0x7fb800168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 >> 172.21.15.118:0/2990283069 conn(0x7fb800067e60 msgr2=0x7fb800154760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 shutdown_connections 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:19.243+0000 7fb80739d700 1 -- 172.21.15.118:0/2990283069 wait complete. 2024-08-18T13:45:19.257 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:19.259 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:45:21.261 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f199449d700 1 Processor -- start 2024-08-18T13:45:21.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f199449d700 1 -- start start 2024-08-18T13:45:21.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f199449d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c14bb00 0x7f198c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f199449d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f198c068530 0x7f198c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f199449d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0673d0 con 0x7f198c06b4a0 2024-08-18T13:45:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f199449d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c152800 con 0x7f198c068530 2024-08-18T13:45:21.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f199449d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c06bdf0 con 0x7f198c14bb00 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f198c068530 0x7f198c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992239700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c14bb00 0x7f198c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f198c068530 0x7f198c06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:57028/0 (socket says 172.21.15.118:57028) 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 -- 172.21.15.118:0/57573799 learned_addr learned my addr 172.21.15.118:0/57573799 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992239700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c14bb00 0x7f198c1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:36530/0 (socket says 172.21.15.118:36530) 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1991a38700 1 --1- 172.21.15.118:0/57573799 >> v1:172.21.15.178:6789/0 conn(0x7f198c06b4a0 0x7f198c067e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:37618/0 (socket says 172.21.15.118:37618) 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 -- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c14bb00 msgr2=0x7f198c1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 --2- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c14bb00 0x7f198c1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 -- 172.21.15.118:0/57573799 >> v1:172.21.15.178:6789/0 conn(0x7f198c06b4a0 legacy=0x7f198c067e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 -- 172.21.15.118:0/57573799 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c067990 con 0x7f198c068530 2024-08-18T13:45:21.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992239700 1 --2- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c14bb00 0x7f198c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:45:21.377 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1992a3a700 1 --2- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c06b8b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1984009f50 tx=0x7f1984009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ca4c8e9fa7c209 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:21.377 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1991237700 1 -- 172.21.15.118:0/57573799 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198400e070 con 0x7f198c068530 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1991237700 1 -- 172.21.15.118:0/57573799 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198400b9c0 con 0x7f198c068530 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.371+0000 7f1991237700 1 -- 172.21.15.118:0/57573799 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840092e0 con 0x7f198c068530 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 -- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 msgr2=0x7f198c06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 --2- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c06b8b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1984009f50 tx=0x7f1984009840 comp rx=0 tx=0).stop 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 -- 172.21.15.118:0/57573799 shutdown_connections 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 --2- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c06b8b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 --2- 172.21.15.118:0/57573799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c14bb00 0x7f198c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 -- 172.21.15.118:0/57573799 >> 172.21.15.118:0/57573799 conn(0x7f198c0c8cc0 msgr2=0x7f198c0c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 -- 172.21.15.118:0/57573799 shutdown_connections 2024-08-18T13:45:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 -- 172.21.15.118:0/57573799 wait complete. 2024-08-18T13:45:21.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 Processor -- start 2024-08-18T13:45:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.375+0000 7f199449d700 1 -- start start 2024-08-18T13:45:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c06b4a0 0x7f198c13f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f198c14bb00 0x7f198c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c138020 con 0x7f198c14bb00 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1388c0 con 0x7f198c068530 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0c7be0 con 0x7f198c06b4a0 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f198c14bb00 0x7f198c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992239700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f198c14bb00 0x7f198c06e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:46740/0 (socket says 172.21.15.118:46740) 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992239700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c138d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:57034/0 (socket says 172.21.15.118:57034) 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 -- 172.21.15.118:0/831793136 learned_addr learned my addr 172.21.15.118:0/831793136 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1991a38700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c06b4a0 0x7f198c13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 -- 172.21.15.118:0/831793136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c06b4a0 msgr2=0x7f198c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c06b4a0 0x7f198c13f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 -- 172.21.15.118:0/831793136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 msgr2=0x7f198c138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c138d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 -- 172.21.15.118:0/831793136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988009420 con 0x7f198c14bb00 2024-08-18T13:45:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f1992a3a700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f198c14bb00 0x7f198c06e7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f198c06d3a0 tx=0x7f1984018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f19827fc700 1 -- 172.21.15.118:0/831793136 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984020070 con 0x7f198c14bb00 2024-08-18T13:45:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f19827fc700 1 -- 172.21.15.118:0/831793136 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198401bb70 con 0x7f198c14bb00 2024-08-18T13:45:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1984009ae0 con 0x7f198c14bb00 2024-08-18T13:45:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f19827fc700 1 -- 172.21.15.118:0/831793136 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984025470 con 0x7f198c14bb00 2024-08-18T13:45:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c138270 con 0x7f198c14bb00 2024-08-18T13:45:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.379+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c0648f0 con 0x7f198c14bb00 2024-08-18T13:45:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.383+0000 7f19827fc700 1 -- 172.21.15.118:0/831793136 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1984003040 con 0x7f198c14bb00 2024-08-18T13:45:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.383+0000 7f19827fc700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1970060860 0x7f1970062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.383+0000 7f1992239700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1970060860 0x7f1970062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.383+0000 7f19827fc700 1 -- 172.21.15.118:0/831793136 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f1984080230 con 0x7f198c14bb00 2024-08-18T13:45:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.383+0000 7f1992239700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1970060860 0x7f1970062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1988009ea0 tx=0x7f19880091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:21.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.387+0000 7f19827fc700 1 -- 172.21.15.118:0/831793136 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1984050220 con 0x7f198c14bb00 2024-08-18T13:45:21.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.591+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198c0c2440 con 0x7f198c14bb00 2024-08-18T13:45:21.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.591+0000 7f19827fc700 1 -- 172.21.15.118:0/831793136 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f1984053840 con 0x7f198c14bb00 2024-08-18T13:45:21.596 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:21.596 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1970060860 msgr2=0x7f1970062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1970060860 0x7f1970062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1988009ea0 tx=0x7f19880091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f198c14bb00 msgr2=0x7f198c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f198c14bb00 0x7f198c06e7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f198c06d3a0 tx=0x7f1984018000 comp rx=0 tx=0).stop 2024-08-18T13:45:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 shutdown_connections 2024-08-18T13:45:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1970060860 0x7f1970062d10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f198c14bb00 0x7f198c06e7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f198c06b4a0 0x7f198c13f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 --2- 172.21.15.118:0/831793136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f198c068530 0x7f198c138d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:21.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 >> 172.21.15.118:0/831793136 conn(0x7f198c0c8cc0 msgr2=0x7f198c141bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:21.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 shutdown_connections 2024-08-18T13:45:21.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:21.595+0000 7f199449d700 1 -- 172.21.15.118:0/831793136 wait complete. 2024-08-18T13:45:21.601 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:21.613 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:45:23.613 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:23.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.727+0000 7f9fa35da700 1 Processor -- start 2024-08-18T13:45:23.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.727+0000 7f9fa35da700 1 -- start start 2024-08-18T13:45:23.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9f9c14c650 0x7f9f9c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:23.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c06c780 con 0x7f9f9c0c8e90 2024-08-18T13:45:23.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c06ccb0 con 0x7f9f9c155920 2024-08-18T13:45:23.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c06a130 con 0x7f9f9c14c650 2024-08-18T13:45:23.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa1b77700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f9f9c0c8e90 0x7f9f9c15cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:44136/0 (socket says 172.21.15.118:44136) 2024-08-18T13:45:23.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa1b77700 1 -- 172.21.15.118:0/3522485018 learned_addr learned my addr 172.21.15.118:0/3522485018 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:45:23.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa0b75700 1 --2- 172.21.15.118:0/3522485018 >> v2:172.21.15.178:3300/0 conn(0x7f9f9c14c650 0x7f9f9c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:23.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317844890 0 0) 0x7f9f9c06c780 con 0x7f9f9c0c8e90 2024-08-18T13:45:23.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f88003570 con 0x7f9f9c0c8e90 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473633610 0 0) 0x7f9f9c06ccb0 con 0x7f9f9c155920 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f9c06c780 con 0x7f9f9c155920 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1592727080 0 0) 0x7f9f88003570 con 0x7f9f9c0c8e90 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa0b75700 1 -- 172.21.15.118:0/3522485018 >> v1:172.21.15.178:6789/0 conn(0x7f9f9c155920 legacy=0x7f9f9c14bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa0b75700 1 -- 172.21.15.118:0/3522485018 >> v1:172.21.15.118:6789/0 conn(0x7f9f9c0c8e90 legacy=0x7f9f9c15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa0b75700 1 -- 172.21.15.118:0/3522485018 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c1563a0 con 0x7f9f9c14c650 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa0b75700 1 --2- 172.21.15.118:0/3522485018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c14c650 0x7f9f9c155cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0098e0 tx=0x7f9f8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4df2a32fb1d1276 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f940017f0 con 0x7f9f9c14c650 2024-08-18T13:45:23.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c011420 con 0x7f9f9c14c650 2024-08-18T13:45:23.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9f93fff700 1 -- 172.21.15.118:0/3522485018 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0117c0 con 0x7f9f9c14c650 2024-08-18T13:45:23.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- 172.21.15.118:0/3522485018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c14c650 msgr2=0x7f9f9c155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:23.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 --2- 172.21.15.118:0/3522485018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c14c650 0x7f9f9c155cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0098e0 tx=0x7f9f8c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- 172.21.15.118:0/3522485018 shutdown_connections 2024-08-18T13:45:23.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 --2- 172.21.15.118:0/3522485018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c14c650 0x7f9f9c155cf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- 172.21.15.118:0/3522485018 >> 172.21.15.118:0/3522485018 conn(0x7f9f9c0c8380 msgr2=0x7f9f9c0c8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:23.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- 172.21.15.118:0/3522485018 shutdown_connections 2024-08-18T13:45:23.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.731+0000 7f9fa35da700 1 -- 172.21.15.118:0/3522485018 wait complete. 2024-08-18T13:45:23.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 Processor -- start 2024-08-18T13:45:23.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 -- start start 2024-08-18T13:45:23.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f9c0c8e90 0x7f9f9c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f9c14c650 0x7f9f9c166720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c155920 0x7f9f9c166c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c14d1e0 con 0x7f9f9c14c650 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c13ff50 con 0x7f9f9c155920 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c0c7e90 con 0x7f9f9c0c8e90 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f9c14c650 0x7f9f9c166720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f9c14c650 0x7f9f9c166720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:46750/0 (socket says 172.21.15.118:46750) 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 -- 172.21.15.118:0/1436691790 learned_addr learned my addr 172.21.15.118:0/1436691790 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:23.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa1376700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f9c0c8e90 0x7f9f9c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:23.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa1b77700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c155920 0x7f9f9c166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:23.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 -- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f9c0c8e90 msgr2=0x7f9f9c14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:23.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f9c0c8e90 0x7f9f9c14d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 -- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c155920 msgr2=0x7f9f9c166c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:23.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c155920 0x7f9f9c166c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 -- 172.21.15.118:0/1436691790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c009420 con 0x7f9f9c14c650 2024-08-18T13:45:23.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa0b75700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f9c14c650 0x7f9f9c166720 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c002000 tx=0x7f9f8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:23.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9f917fa700 1 -- 172.21.15.118:0/1436691790 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c000d10 con 0x7f9f9c14c650 2024-08-18T13:45:23.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9f917fa700 1 -- 172.21.15.118:0/1436691790 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c01cb30 con 0x7f9f9c14c650 2024-08-18T13:45:23.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c1401a0 con 0x7f9f9c14c650 2024-08-18T13:45:23.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c140760 con 0x7f9f9c14c650 2024-08-18T13:45:23.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.735+0000 7f9f917fa700 1 -- 172.21.15.118:0/1436691790 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c01a450 con 0x7f9f9c14c650 2024-08-18T13:45:23.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.739+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c0648f0 con 0x7f9f9c14c650 2024-08-18T13:45:23.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.739+0000 7f9f917fa700 1 -- 172.21.15.118:0/1436691790 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9f8c01a880 con 0x7f9f9c14c650 2024-08-18T13:45:23.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.739+0000 7f9f917fa700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9f80060860 0x7f9f80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:23.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.739+0000 7f9f917fa700 1 -- 172.21.15.118:0/1436691790 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f9f8c07e6d0 con 0x7f9f9c14c650 2024-08-18T13:45:23.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.739+0000 7f9fa1376700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9f80060860 0x7f9f80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:23.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.739+0000 7f9fa1376700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9f80060860 0x7f9f80062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f98007fd0 tx=0x7f9f98007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.747+0000 7f9f917fa700 1 -- 172.21.15.118:0/1436691790 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9f8c04e6c0 con 0x7f9f9c14c650 2024-08-18T13:45:23.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.947+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f9c0c0ed0 con 0x7f9f9c14c650 2024-08-18T13:45:23.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9f917fa700 1 -- 172.21.15.118:0/1436691790 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f9f8c051ce0 con 0x7f9f9c14c650 2024-08-18T13:45:23.954 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:23.954 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:23.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9f80060860 msgr2=0x7f9f80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:23.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9f80060860 0x7f9f80062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f98007fd0 tx=0x7f9f98007d50 comp rx=0 tx=0).stop 2024-08-18T13:45:23.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f9c14c650 msgr2=0x7f9f9c166720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:23.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f9c14c650 0x7f9f9c166720 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c002000 tx=0x7f9f8c016000 comp rx=0 tx=0).stop 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 shutdown_connections 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9f80060860 0x7f9f80062d10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f9c155920 0x7f9f9c166c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f9c14c650 0x7f9f9c166720 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 --2- 172.21.15.118:0/1436691790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f9c0c8e90 0x7f9f9c14d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 >> 172.21.15.118:0/1436691790 conn(0x7f9f9c0c8380 msgr2=0x7f9f9c0681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 shutdown_connections 2024-08-18T13:45:23.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:23.951+0000 7f9fa35da700 1 -- 172.21.15.118:0/1436691790 wait complete. 2024-08-18T13:45:23.958 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:23.970 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:45:25.971 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:26.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.083+0000 7f66d28c1700 1 Processor -- start 2024-08-18T13:45:26.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.083+0000 7f66d28c1700 1 -- start start 2024-08-18T13:45:26.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f66cc0bf8c0 0x7f66cc152400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:26.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66cc06bc40 0x7f66cc156390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:26.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc155f00 con 0x7f66cc069560 2024-08-18T13:45:26.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66cc0beae0 con 0x7f66cc0bf8c0 2024-08-18T13:45:26.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66cc155980 con 0x7f66cc06bc40 2024-08-18T13:45:26.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cbfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f66cc0bf8c0 0x7f66cc152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:26.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d0e5e700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f66cc069560 0x7f66cc14c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:58246/0 (socket says 172.21.15.118:58246) 2024-08-18T13:45:26.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d0e5e700 1 -- 172.21.15.118:0/2190632230 learned_addr learned my addr 172.21.15.118:0/2190632230 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:45:26.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cb7fe700 1 --2- 172.21.15.118:0/2190632230 >> v2:172.21.15.178:3300/0 conn(0x7f66cc06bc40 0x7f66cc156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:26.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cbfff700 1 -- 172.21.15.118:0/2190632230 >> v2:172.21.15.178:3300/0 conn(0x7f66cc06bc40 msgr2=0x7f66cc156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:26.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cbfff700 1 --2- 172.21.15.118:0/2190632230 >> v2:172.21.15.178:3300/0 conn(0x7f66cc06bc40 0x7f66cc156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cbfff700 1 -- 172.21.15.118:0/2190632230 >> v1:172.21.15.118:6789/0 conn(0x7f66cc069560 legacy=0x7f66cc14c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:26.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cbfff700 1 -- 172.21.15.118:0/2190632230 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0c6060 con 0x7f66cc0bf8c0 2024-08-18T13:45:26.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cb7fe700 1 --2- 172.21.15.118:0/2190632230 >> v2:172.21.15.178:3300/0 conn(0x7f66cc06bc40 0x7f66cc156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:45:26.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66cbfff700 1 --2- 172.21.15.118:0/2190632230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc0bf8c0 0x7f66cc152400 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f66cc155ac0 tx=0x7f66c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c6e0cf7e897a1e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:26.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66caffd700 1 -- 172.21.15.118:0/2190632230 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4016070 con 0x7f66cc0bf8c0 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66caffd700 1 -- 172.21.15.118:0/2190632230 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c400bc00 con 0x7f66cc0bf8c0 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66caffd700 1 -- 172.21.15.118:0/2190632230 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400e070 con 0x7f66cc0bf8c0 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- 172.21.15.118:0/2190632230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc0bf8c0 msgr2=0x7f66cc152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 --2- 172.21.15.118:0/2190632230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc0bf8c0 0x7f66cc152400 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f66cc155ac0 tx=0x7f66c4009780 comp rx=0 tx=0).stop 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- 172.21.15.118:0/2190632230 shutdown_connections 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 --2- 172.21.15.118:0/2190632230 >> v2:172.21.15.178:3300/0 conn(0x7f66cc06bc40 0x7f66cc156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 --2- 172.21.15.118:0/2190632230 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc0bf8c0 0x7f66cc152400 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- 172.21.15.118:0/2190632230 >> 172.21.15.118:0/2190632230 conn(0x7f66cc0c9930 msgr2=0x7f66cc0c5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- 172.21.15.118:0/2190632230 shutdown_connections 2024-08-18T13:45:26.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.087+0000 7f66d28c1700 1 -- 172.21.15.118:0/2190632230 wait complete. 2024-08-18T13:45:26.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 Processor -- start 2024-08-18T13:45:26.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 -- start start 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66cc069560 0x7f66cc0c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc06bc40 0x7f66cc150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66cc0c1d50 0x7f66cc151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc0c5680 con 0x7f66cc06bc40 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc0c12e0 con 0x7f66cc069560 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d28c1700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc1527b0 con 0x7f66cc0c1d50 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc06bc40 0x7f66cc150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:26.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc06bc40 0x7f66cc150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39588/0 (socket says 172.21.15.118:39588) 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 -- 172.21.15.118:0/122380238 learned_addr learned my addr 172.21.15.118:0/122380238 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66d0e5e700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66cc0c1d50 0x7f66cc151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cbfff700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66cc069560 0x7f66cc0c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 -- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66cc0c1d50 msgr2=0x7f66cc151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66cc0c1d50 0x7f66cc151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 -- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66cc069560 msgr2=0x7f66cc0c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66cc069560 0x7f66cc0c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 -- 172.21.15.118:0/122380238 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4009a20 con 0x7f66cc06bc40 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.091+0000 7f66cb7fe700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc06bc40 0x7f66cc150f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00d890 tx=0x7f66bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66b3fff700 1 -- 172.21.15.118:0/122380238 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0125f0 con 0x7f66cc06bc40 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66b3fff700 1 -- 172.21.15.118:0/122380238 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc012750 con 0x7f66cc06bc40 2024-08-18T13:45:26.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc1507b0 con 0x7f66cc06bc40 2024-08-18T13:45:26.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66b3fff700 1 -- 172.21.15.118:0/122380238 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc012af0 con 0x7f66cc06bc40 2024-08-18T13:45:26.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc149b30 con 0x7f66cc06bc40 2024-08-18T13:45:26.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66cc150c40 con 0x7f66cc06bc40 2024-08-18T13:45:26.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66b3fff700 1 -- 172.21.15.118:0/122380238 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f66bc014430 con 0x7f66cc06bc40 2024-08-18T13:45:26.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66b3fff700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060860 0x7f66b8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:26.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66b3fff700 1 -- 172.21.15.118:0/122380238 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f66bc07d380 con 0x7f66cc06bc40 2024-08-18T13:45:26.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66cbfff700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060860 0x7f66b8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:26.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.095+0000 7f66cbfff700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060860 0x7f66b8062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66cc06e6c0 tx=0x7f66c4018000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:26.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.103+0000 7f66b3fff700 1 -- 172.21.15.118:0/122380238 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f66cc150c40 con 0x7f66cc06bc40 2024-08-18T13:45:26.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.323+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66cc150c40 con 0x7f66cc06bc40 2024-08-18T13:45:26.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.323+0000 7f66b3fff700 1 -- 172.21.15.118:0/122380238 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f66cc150c40 con 0x7f66cc06bc40 2024-08-18T13:45:26.327 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:26.327 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:26.330 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060860 msgr2=0x7f66b8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:26.330 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060860 0x7f66b8062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66cc06e6c0 tx=0x7f66c4018000 comp rx=0 tx=0).stop 2024-08-18T13:45:26.330 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc06bc40 msgr2=0x7f66cc150f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:26.330 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc06bc40 0x7f66cc150f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00d890 tx=0x7f66bc008040 comp rx=0 tx=0).stop 2024-08-18T13:45:26.330 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 shutdown_connections 2024-08-18T13:45:26.330 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f66b8060860 0x7f66b8062d10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66cc0c1d50 0x7f66cc151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66cc06bc40 0x7f66cc150f20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 --2- 172.21.15.118:0/122380238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66cc069560 0x7f66cc0c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:26.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 >> 172.21.15.118:0/122380238 conn(0x7f66cc0c9930 msgr2=0x7f66cc1529f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:26.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 shutdown_connections 2024-08-18T13:45:26.331 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:26.327+0000 7f66d28c1700 1 -- 172.21.15.118:0/122380238 wait complete. 2024-08-18T13:45:26.332 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:26.346 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:45:28.348 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:28.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.455+0000 7f5424af2700 1 Processor -- start 2024-08-18T13:45:28.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- start start 2024-08-18T13:45:28.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f542014bb00 0x7f5420152370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:28.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f54200c2c60 0x7f54200673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:28.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f542005e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54200c2520 con 0x7f542014bb00 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54201528b0 con 0x7f54200c2c60 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200c5520 con 0x7f54200c3070 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f542014bb00 0x7f5420152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f542014bb00 0x7f5420152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39604/0 (socket says 172.21.15.118:39604) 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 -- 172.21.15.118:0/1256674206 learned_addr learned my addr 172.21.15.118:0/1256674206 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541dd9b700 1 --2- 172.21.15.118:0/1256674206 >> v2:172.21.15.178:3300/0 conn(0x7f54200c2c60 0x7f54200673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541ed9d700 1 --2- 172.21.15.118:0/1256674206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f542005e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 -- 172.21.15.118:0/1256674206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 msgr2=0x7f542005e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 --2- 172.21.15.118:0/1256674206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f542005e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 -- 172.21.15.118:0/1256674206 >> v2:172.21.15.178:3300/0 conn(0x7f54200c2c60 msgr2=0x7f54200673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 --2- 172.21.15.118:0/1256674206 >> v2:172.21.15.178:3300/0 conn(0x7f54200c2c60 0x7f54200673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.464 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 -- 172.21.15.118:0/1256674206 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542006e350 con 0x7f542014bb00 2024-08-18T13:45:28.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541e59c700 1 --2- 172.21.15.118:0/1256674206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f5420152370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5420065560 tx=0x7f5410009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7020a8529f949a9b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:28.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541d59a700 1 -- 172.21.15.118:0/1256674206 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000b640 con 0x7f542014bb00 2024-08-18T13:45:28.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541d59a700 1 -- 172.21.15.118:0/1256674206 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541000b7a0 con 0x7f542014bb00 2024-08-18T13:45:28.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f541d59a700 1 -- 172.21.15.118:0/1256674206 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000bb60 con 0x7f542014bb00 2024-08-18T13:45:28.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- 172.21.15.118:0/1256674206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 msgr2=0x7f5420152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 --2- 172.21.15.118:0/1256674206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f5420152370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5420065560 tx=0x7f5410009250 comp rx=0 tx=0).stop 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- 172.21.15.118:0/1256674206 shutdown_connections 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 --2- 172.21.15.118:0/1256674206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f542005e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 --2- 172.21.15.118:0/1256674206 >> v2:172.21.15.178:3300/0 conn(0x7f54200c2c60 0x7f54200673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 --2- 172.21.15.118:0/1256674206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f5420152370 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- 172.21.15.118:0/1256674206 >> 172.21.15.118:0/1256674206 conn(0x7f54200c8cf0 msgr2=0x7f54200c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- 172.21.15.118:0/1256674206 shutdown_connections 2024-08-18T13:45:28.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.459+0000 7f5424af2700 1 -- 172.21.15.118:0/1256674206 wait complete. 2024-08-18T13:45:28.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 Processor -- start 2024-08-18T13:45:28.467 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 -- start start 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54200c2c60 0x7f5420149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f5420141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f54201640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542014f0c0 con 0x7f542014bb00 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420163c60 con 0x7f54200c2c60 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f5424af2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542014a430 con 0x7f54200c3070 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f54201640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:28.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541dd9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f5420141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f54201640f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39614/0 (socket says 172.21.15.118:39614) 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 -- 172.21.15.118:0/3881806290 learned_addr learned my addr 172.21.15.118:0/3881806290 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541e59c700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54200c2c60 0x7f5420149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 -- 172.21.15.118:0/3881806290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 msgr2=0x7f5420141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f5420141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 -- 172.21.15.118:0/3881806290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54200c2c60 msgr2=0x7f5420149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54200c2c60 0x7f5420149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 -- 172.21.15.118:0/3881806290 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100094f0 con 0x7f542014bb00 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.463+0000 7f541ed9d700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f54201640f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f541800e380 tx=0x7f541800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f540e7fc700 1 -- 172.21.15.118:0/3881806290 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800eec0 con 0x7f542014bb00 2024-08-18T13:45:28.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f540e7fc700 1 -- 172.21.15.118:0/3881806290 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418012430 con 0x7f542014bb00 2024-08-18T13:45:28.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f540e7fc700 1 -- 172.21.15.118:0/3881806290 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180127d0 con 0x7f542014bb00 2024-08-18T13:45:28.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542014b1f0 con 0x7f542014bb00 2024-08-18T13:45:28.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54201638f0 con 0x7f542014bb00 2024-08-18T13:45:28.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54200648f0 con 0x7f542014bb00 2024-08-18T13:45:28.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f540e7fc700 1 -- 172.21.15.118:0/3881806290 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5418012b40 con 0x7f542014bb00 2024-08-18T13:45:28.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f540e7fc700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5400060860 0x7f5400062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:28.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f540e7fc700 1 -- 172.21.15.118:0/3881806290 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f5418016070 con 0x7f542014bb00 2024-08-18T13:45:28.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.467+0000 7f541e59c700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5400060860 0x7f5400062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:28.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.471+0000 7f541e59c700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5400060860 0x7f5400062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5420065560 tx=0x7f5410017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:28.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.475+0000 7f540e7fc700 1 -- 172.21.15.118:0/3881806290 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f541804fe90 con 0x7f542014bb00 2024-08-18T13:45:28.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.695+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f542006f410 con 0x7f542014bb00 2024-08-18T13:45:28.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.695+0000 7f540e7fc700 1 -- 172.21.15.118:0/3881806290 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f54180534b0 con 0x7f542014bb00 2024-08-18T13:45:28.699 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:28.699 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5400060860 msgr2=0x7f5400062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5400060860 0x7f5400062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5420065560 tx=0x7f5410017000 comp rx=0 tx=0).stop 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 msgr2=0x7f54201640f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f54201640f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f541800e380 tx=0x7f541800b040 comp rx=0 tx=0).stop 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 shutdown_connections 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5400060860 0x7f5400062d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f542014bb00 0x7f54201640f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54200c3070 0x7f5420141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 --2- 172.21.15.118:0/3881806290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54200c2c60 0x7f5420149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:28.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 >> 172.21.15.118:0/3881806290 conn(0x7f54200c8cf0 msgr2=0x7f542006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:28.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 shutdown_connections 2024-08-18T13:45:28.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:28.699+0000 7f5424af2700 1 -- 172.21.15.118:0/3881806290 wait complete. 2024-08-18T13:45:28.704 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:28.717 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:45:30.718 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:30.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.835+0000 7f48e4d35700 1 Processor -- start 2024-08-18T13:45:30.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.835+0000 7f48e4d35700 1 -- start start 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48e006c570 0x7f48e006c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e00c92f0 0x7f48e0068690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e00c9830 con 0x7f48e006ce80 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e0153370 con 0x7f48e006c570 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e01524e0 con 0x7f48e00c92f0 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e00c92f0 0x7f48e0068690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e00c92f0 0x7f48e0068690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:60262/0 (socket says 172.21.15.118:60262) 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 -- 172.21.15.118:0/2171566678 learned_addr learned my addr 172.21.15.118:0/2171566678 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:30.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48de59c700 1 --2- 172.21.15.118:0/2171566678 >> v2:172.21.15.178:3300/0 conn(0x7f48e006c570 0x7f48e006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 -- 172.21.15.118:0/2171566678 >> v2:172.21.15.178:3300/0 conn(0x7f48e006c570 msgr2=0x7f48e006c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:30.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 --2- 172.21.15.118:0/2171566678 >> v2:172.21.15.178:3300/0 conn(0x7f48e006c570 0x7f48e006c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 -- 172.21.15.118:0/2171566678 >> v1:172.21.15.178:6789/0 conn(0x7f48e006ce80 legacy=0x7f48e0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 -- 172.21.15.118:0/2171566678 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e01526f0 con 0x7f48e00c92f0 2024-08-18T13:45:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48d5d9b700 1 --2- 172.21.15.118:0/2171566678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e00c92f0 0x7f48e0068690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48d00099b0 tx=0x7f48d0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510e6f1dd109d26a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48ddd9b700 1 -- 172.21.15.118:0/2171566678 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0009ea0 con 0x7f48e00c92f0 2024-08-18T13:45:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48ddd9b700 1 -- 172.21.15.118:0/2171566678 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d000b860 con 0x7f48e00c92f0 2024-08-18T13:45:30.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48ddd9b700 1 -- 172.21.15.118:0/2171566678 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000bc00 con 0x7f48e00c92f0 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- 172.21.15.118:0/2171566678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e00c92f0 msgr2=0x7f48e0068690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 --2- 172.21.15.118:0/2171566678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e00c92f0 0x7f48e0068690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48d00099b0 tx=0x7f48d0009250 comp rx=0 tx=0).stop 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- 172.21.15.118:0/2171566678 shutdown_connections 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 --2- 172.21.15.118:0/2171566678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e00c92f0 0x7f48e0068690 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 --2- 172.21.15.118:0/2171566678 >> v2:172.21.15.178:3300/0 conn(0x7f48e006c570 0x7f48e006c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- 172.21.15.118:0/2171566678 >> 172.21.15.118:0/2171566678 conn(0x7f48e006f6d0 msgr2=0x7f48e00c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- 172.21.15.118:0/2171566678 shutdown_connections 2024-08-18T13:45:30.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.839+0000 7f48e4d35700 1 -- 172.21.15.118:0/2171566678 wait complete. 2024-08-18T13:45:30.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 Processor -- start 2024-08-18T13:45:30.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 -- start start 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e006c570 0x7f48e0162d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e006ce80 0x7f48e00c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48e00c92f0 0x7f48e00c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e00c38f0 con 0x7f48e006ce80 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e00c6490 con 0x7f48e00c92f0 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48e4d35700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e0148280 con 0x7f48e006c570 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e006ce80 0x7f48e00c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e006ce80 0x7f48e00c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39626/0 (socket says 172.21.15.118:39626) 2024-08-18T13:45:30.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 -- 172.21.15.118:0/1856619969 learned_addr learned my addr 172.21.15.118:0/1856619969 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:30.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48de59c700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e006c570 0x7f48e0162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:30.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48ded9d700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48e00c92f0 0x7f48e00c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:30.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 -- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e006c570 msgr2=0x7f48e0162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:30.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e006c570 0x7f48e0162d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:30.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 -- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48e00c92f0 msgr2=0x7f48e00c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:30.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48e00c92f0 0x7f48e00c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:30.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 -- 172.21.15.118:0/1856619969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00094f0 con 0x7f48e006ce80 2024-08-18T13:45:30.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d5d9b700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e006ce80 0x7f48e00c5f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009290 tx=0x7f48d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:30.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d6ffd700 1 -- 172.21.15.118:0/1856619969 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0000d10 con 0x7f48e006ce80 2024-08-18T13:45:30.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.843+0000 7f48d6ffd700 1 -- 172.21.15.118:0/1856619969 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d000bbe0 con 0x7f48e006ce80 2024-08-18T13:45:30.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e0148560 con 0x7f48e006ce80 2024-08-18T13:45:30.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48d6ffd700 1 -- 172.21.15.118:0/1856619969 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d001a660 con 0x7f48e006ce80 2024-08-18T13:45:30.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e00c32a0 con 0x7f48e006ce80 2024-08-18T13:45:30.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e00648f0 con 0x7f48e006ce80 2024-08-18T13:45:30.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48d6ffd700 1 -- 172.21.15.118:0/1856619969 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f48d001aa90 con 0x7f48e006ce80 2024-08-18T13:45:30.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48d6ffd700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f48c0060860 0x7f48c0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:30.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48d6ffd700 1 -- 172.21.15.118:0/1856619969 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f48d007e860 con 0x7f48e006ce80 2024-08-18T13:45:30.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48de59c700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f48c0060860 0x7f48c0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:30.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.847+0000 7f48de59c700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f48c0060860 0x7f48c0062d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009d10 tx=0x7f48c80091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:30.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:30.855+0000 7f48d6ffd700 1 -- 172.21.15.118:0/1856619969 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f48d004e900 con 0x7f48e006ce80 2024-08-18T13:45:31.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48e00c3a30 con 0x7f48e006ce80 2024-08-18T13:45:31.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48d6ffd700 1 -- 172.21.15.118:0/1856619969 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f48e00c3a30 con 0x7f48e006ce80 2024-08-18T13:45:31.071 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:31.071 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:31.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f48c0060860 msgr2=0x7f48c0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:31.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f48c0060860 0x7f48c0062d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009d10 tx=0x7f48c80091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:31.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e006ce80 msgr2=0x7f48e00c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:31.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e006ce80 0x7f48e00c5f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009290 tx=0x7f48d0017000 comp rx=0 tx=0).stop 2024-08-18T13:45:31.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 shutdown_connections 2024-08-18T13:45:31.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f48c0060860 0x7f48c0062d10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:31.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48e00c92f0 0x7f48e00c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:31.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e006ce80 0x7f48e00c5f50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:31.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 --2- 172.21.15.118:0/1856619969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48e006c570 0x7f48e0162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:31.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 >> 172.21.15.118:0/1856619969 conn(0x7f48e006f6d0 msgr2=0x7f48e014f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:31.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.067+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 shutdown_connections 2024-08-18T13:45:31.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:31.071+0000 7f48e4d35700 1 -- 172.21.15.118:0/1856619969 wait complete. 2024-08-18T13:45:31.077 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:31.084 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:45:33.085 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:33.201 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 Processor -- start 2024-08-18T13:45:33.202 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 -- start start 2024-08-18T13:45:33.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa0280bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:33.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa02806c810 0x7fa0280c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:33.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0280c3080 con 0x7fa0280bb8e0 2024-08-18T13:45:33.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0280c4cb0 con 0x7fa02806c810 2024-08-18T13:45:33.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02806cc90 con 0x7fa0280c8e90 2024-08-18T13:45:33.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa026ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fa0280bb8e0 0x7fa02806b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:58262/0 (socket says 172.21.15.118:58262) 2024-08-18T13:45:33.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa026ffd700 1 -- 172.21.15.118:0/506492753 learned_addr learned my addr 172.21.15.118:0/506492753 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa0277fe700 1 --2- 172.21.15.118:0/506492753 >> v2:172.21.15.118:3300/0 conn(0x7fa02806c810 0x7fa0280c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa0267fc700 1 --2- 172.21.15.118:0/506492753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa0280bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa025ffb700 1 -- 172.21.15.118:0/506492753 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625301170 0 0) 0x7fa0280c3080 con 0x7fa0280bb8e0 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa0277fe700 1 -- 172.21.15.118:0/506492753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 msgr2=0x7fa0280bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa0277fe700 1 --2- 172.21.15.118:0/506492753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa0280bbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa0277fe700 1 -- 172.21.15.118:0/506492753 >> v1:172.21.15.118:6789/0 conn(0x7fa0280bb8e0 legacy=0x7fa02806b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa0277fe700 1 -- 172.21.15.118:0/506492753 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028156580 con 0x7fa02806c810 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa0277fe700 1 --2- 172.21.15.118:0/506492753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02806c810 0x7fa0280c3510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa0200098e0 tx=0x7fa0200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989a9f9ebc62e90a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa025ffb700 1 -- 172.21.15.118:0/506492753 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000b640 con 0x7fa02806c810 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa025ffb700 1 -- 172.21.15.118:0/506492753 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000b7a0 con 0x7fa02806c810 2024-08-18T13:45:33.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa025ffb700 1 -- 172.21.15.118:0/506492753 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000bb40 con 0x7fa02806c810 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 -- 172.21.15.118:0/506492753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02806c810 msgr2=0x7fa0280c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 --2- 172.21.15.118:0/506492753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02806c810 0x7fa0280c3510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa0200098e0 tx=0x7fa0200091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 -- 172.21.15.118:0/506492753 shutdown_connections 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 --2- 172.21.15.118:0/506492753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa02806c810 0x7fa0280c3510 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 --2- 172.21.15.118:0/506492753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa0280bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.199+0000 7fa02d35b700 1 -- 172.21.15.118:0/506492753 >> 172.21.15.118:0/506492753 conn(0x7fa0280c8380 msgr2=0x7fa0280c8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 -- 172.21.15.118:0/506492753 shutdown_connections 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 -- 172.21.15.118:0/506492753 wait complete. 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 Processor -- start 2024-08-18T13:45:33.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 -- start start 2024-08-18T13:45:33.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa02806c810 0x7fa0280c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:33.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0280bb8e0 0x7fa028167c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:33.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa02814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:33.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02805ebf0 con 0x7fa0280bb8e0 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028165580 con 0x7fa02806c810 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa02d35b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280c7870 con 0x7fa0280c8e90 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0280bb8e0 0x7fa028167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0280bb8e0 0x7fa028167c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39652/0 (socket says 172.21.15.118:39652) 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 -- 172.21.15.118:0/2762485280 learned_addr learned my addr 172.21.15.118:0/2762485280 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0277fe700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa02814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa026ffd700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa02806c810 0x7fa0280c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 -- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 msgr2=0x7fa02814ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa02814ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 -- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa02806c810 msgr2=0x7fa0280c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa02806c810 0x7fa0280c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 -- 172.21.15.118:0/2762485280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020009420 con 0x7fa0280bb8e0 2024-08-18T13:45:33.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.203+0000 7fa0267fc700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0280bb8e0 0x7fa028167c20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e960 tx=0x7fa0100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:33.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa01effd700 1 -- 172.21.15.118:0/2762485280 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010013680 con 0x7fa0280bb8e0 2024-08-18T13:45:33.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa01effd700 1 -- 172.21.15.118:0/2762485280 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0100137e0 con 0x7fa0280bb8e0 2024-08-18T13:45:33.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0281650f0 con 0x7fa0280bb8e0 2024-08-18T13:45:33.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa01effd700 1 -- 172.21.15.118:0/2762485280 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010013b80 con 0x7fa0280bb8e0 2024-08-18T13:45:33.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0280b8c60 con 0x7fa0280bb8e0 2024-08-18T13:45:33.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0280648f0 con 0x7fa0280bb8e0 2024-08-18T13:45:33.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa01effd700 1 -- 172.21.15.118:0/2762485280 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa01000c470 con 0x7fa0280bb8e0 2024-08-18T13:45:33.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa01effd700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa008060860 0x7fa008062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:33.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa01effd700 1 -- 172.21.15.118:0/2762485280 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fa01007d140 con 0x7fa0280bb8e0 2024-08-18T13:45:33.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa026ffd700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa008060860 0x7fa008062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:33.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.207+0000 7fa026ffd700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa008060860 0x7fa008062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa018002250 tx=0x7fa018001a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:33.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.215+0000 7fa01effd700 1 -- 172.21.15.118:0/2762485280 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa01004d130 con 0x7fa0280bb8e0 2024-08-18T13:45:33.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.419+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0280b86e0 con 0x7fa0280bb8e0 2024-08-18T13:45:33.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.419+0000 7fa01effd700 1 -- 172.21.15.118:0/2762485280 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6127 (secure 0 0 0) 0x7fa010050750 con 0x7fa0280bb8e0 2024-08-18T13:45:33.423 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:33.424 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5209,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:44:34.688528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4805":{"gid":4805,"name":"d","rank":0,"incarnation":16,"state":"up:active","state_seq":31,"addr":"172.21.15.178:6837/3452743394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3452743394},{"type":"v1","addr":"172.21.15.178:6837","nonce":3452743394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa008060860 msgr2=0x7fa008062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa008060860 0x7fa008062d10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa018002250 tx=0x7fa018001a20 comp rx=0 tx=0).stop 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0280bb8e0 msgr2=0x7fa028167c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0280bb8e0 0x7fa028167c20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e960 tx=0x7fa0100091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 shutdown_connections 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa008060860 0x7fa008062d10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0280c8e90 0x7fa02814ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0280bb8e0 0x7fa028167c20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 --2- 172.21.15.118:0/2762485280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa02806c810 0x7fa0280c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:33.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 >> 172.21.15.118:0/2762485280 conn(0x7fa0280c8380 msgr2=0x7fa0280c4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:33.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 shutdown_connections 2024-08-18T13:45:33.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:33.423+0000 7fa02d35b700 1 -- 172.21.15.118:0/2762485280 wait complete. 2024-08-18T13:45:33.427 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 20 2024-08-18T13:45:33.437 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:45:35.438 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:35.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39bca7c700 1 Processor -- start 2024-08-18T13:45:35.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39bca7c700 1 -- start start 2024-08-18T13:45:35.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39bca7c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39b8069810 0x7f39b80673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39bca7c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39bca7c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b80c20c0 con 0x7f39b8069be0 2024-08-18T13:45:35.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39bca7c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b80c2960 con 0x7f39b8069810 2024-08-18T13:45:35.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39bca7c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b8067910 con 0x7f39b8152f40 2024-08-18T13:45:35.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b7fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b77fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39b8069810 0x7f39b80673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b6ffd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f39b8069be0 0x7f39b80c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:43514/0 (socket says 172.21.15.118:43514) 2024-08-18T13:45:35.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b6ffd700 1 -- 172.21.15.118:0/1661506699 learned_addr learned my addr 172.21.15.118:0/1661506699 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:45:35.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b77fe700 1 -- 172.21.15.118:0/1661506699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 msgr2=0x7f39b81533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:35.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b77fe700 1 --2- 172.21.15.118:0/1661506699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b77fe700 1 -- 172.21.15.118:0/1661506699 >> v1:172.21.15.178:6789/0 conn(0x7f39b8069be0 legacy=0x7f39b80c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b77fe700 1 -- 172.21.15.118:0/1661506699 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b80c2df0 con 0x7f39b8069810 2024-08-18T13:45:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.547+0000 7f39b77fe700 1 --2- 172.21.15.118:0/1661506699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b80673d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f39a00098e0 tx=0x7f39a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657119eb32f66f3a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39b67fc700 1 -- 172.21.15.118:0/1661506699 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a000b640 con 0x7f39b8069810 2024-08-18T13:45:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39b67fc700 1 -- 172.21.15.118:0/1661506699 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a000b7a0 con 0x7f39b8069810 2024-08-18T13:45:35.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39b67fc700 1 -- 172.21.15.118:0/1661506699 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a000bb40 con 0x7f39b8069810 2024-08-18T13:45:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 -- 172.21.15.118:0/1661506699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 msgr2=0x7f39b80673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 --2- 172.21.15.118:0/1661506699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b80673d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f39a00098e0 tx=0x7f39a00091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 -- 172.21.15.118:0/1661506699 shutdown_connections 2024-08-18T13:45:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 --2- 172.21.15.118:0/1661506699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 --2- 172.21.15.118:0/1661506699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b80673d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 -- 172.21.15.118:0/1661506699 >> 172.21.15.118:0/1661506699 conn(0x7f39b806f4d0 msgr2=0x7f39b80c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:35.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 -- 172.21.15.118:0/1661506699 shutdown_connections 2024-08-18T13:45:35.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 -- 172.21.15.118:0/1661506699 wait complete. 2024-08-18T13:45:35.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 Processor -- start 2024-08-18T13:45:35.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.551+0000 7f39bca7c700 1 -- start start 2024-08-18T13:45:35.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b8168c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39b8069be0 0x7f39b8144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b81461e0 con 0x7f39b8069be0 2024-08-18T13:45:35.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b814f480 con 0x7f39b8069810 2024-08-18T13:45:35.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b814f720 con 0x7f39b8152f40 2024-08-18T13:45:35.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39b8069be0 0x7f39b8144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b8168c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b8168c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:41526/0 (socket says 172.21.15.118:41526) 2024-08-18T13:45:35.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b77fe700 1 -- 172.21.15.118:0/200723494 learned_addr learned my addr 172.21.15.118:0/200723494 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:35.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39b8069be0 0x7f39b8144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37550/0 (socket says 172.21.15.118:37550) 2024-08-18T13:45:35.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b7fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 -- 172.21.15.118:0/200723494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 msgr2=0x7f39b81448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 -- 172.21.15.118:0/200723494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 msgr2=0x7f39b8168c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b8168c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 -- 172.21.15.118:0/200723494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a0009420 con 0x7f39b8069be0 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39b6ffd700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39b8069be0 0x7f39b8144380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f39a800cc70 tx=0x7f39a8001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39af7fe700 1 -- 172.21.15.118:0/200723494 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8004bb0 con 0x7f39b8069be0 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39af7fe700 1 -- 172.21.15.118:0/200723494 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a8004d10 con 0x7f39b8069be0 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b8169760 con 0x7f39b8069be0 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39af7fe700 1 -- 172.21.15.118:0/200723494 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a800aa20 con 0x7f39b8069be0 2024-08-18T13:45:35.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.555+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b814e790 con 0x7f39b8069be0 2024-08-18T13:45:35.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.559+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b8069160 con 0x7f39b8069be0 2024-08-18T13:45:35.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.559+0000 7f39af7fe700 1 -- 172.21.15.118:0/200723494 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f39a80070f0 con 0x7f39b8069be0 2024-08-18T13:45:35.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.559+0000 7f39af7fe700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3998060860 0x7f3998062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.559+0000 7f39af7fe700 1 -- 172.21.15.118:0/200723494 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f39a807cc20 con 0x7f39b8069be0 2024-08-18T13:45:35.563 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.559+0000 7f39b77fe700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3998060860 0x7f3998062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.564 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.559+0000 7f39b77fe700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3998060860 0x7f3998062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f39a0003080 tx=0x7f39a0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:35.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.567+0000 7f39af7fe700 1 -- 172.21.15.118:0/200723494 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f39b8069160 con 0x7f39b8069be0 2024-08-18T13:45:35.773 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.771+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39b8069160 con 0x7f39b8069be0 2024-08-18T13:45:35.774 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.771+0000 7f39af7fe700 1 -- 172.21.15.118:0/200723494 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5366 (secure 0 0 0) 0x7f39b8069160 con 0x7f39b8069be0 2024-08-18T13:45:35.774 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:35.775 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:35.570401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.771+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3998060860 msgr2=0x7f3998062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.771+0000 7f39bca7c700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3998060860 0x7f3998062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f39a0003080 tx=0x7f39a0017000 comp rx=0 tx=0).stop 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.771+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39b8069be0 msgr2=0x7f39b8144380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.771+0000 7f39bca7c700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39b8069be0 0x7f39b8144380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f39a800cc70 tx=0x7f39a8001a10 comp rx=0 tx=0).stop 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 shutdown_connections 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3998060860 0x7f3998062d10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39b8152f40 0x7f39b81448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39b8069be0 0x7f39b8144380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.777 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 --2- 172.21.15.118:0/200723494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39b8069810 0x7f39b8168c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 >> 172.21.15.118:0/200723494 conn(0x7f39b806f4d0 msgr2=0x7f39b80c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:35.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 shutdown_connections 2024-08-18T13:45:35.778 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.775+0000 7f39bca7c700 1 -- 172.21.15.118:0/200723494 wait complete. 2024-08-18T13:45:35.778 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 21 2024-08-18T13:45:35.793 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2024-08-18T13:45:35.793 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-08-18T13:45:35.793 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:35.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547b59e700 1 Processor -- start 2024-08-18T13:45:35.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547b59e700 1 -- start start 2024-08-18T13:45:35.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547b59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f547c14f410 0x7f547c0ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547b59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f547c14f7e0 0x7f547c0abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547b59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c0c1c80 con 0x7f547c158830 2024-08-18T13:45:35.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547b59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c0c18e0 con 0x7f547c14f410 2024-08-18T13:45:35.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547b59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c0c09f0 con 0x7f547c14f7e0 2024-08-18T13:45:35.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f547c14f410 0x7f547c0ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f547c14f410 0x7f547c0ab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37566/0 (socket says 172.21.15.118:37566) 2024-08-18T13:45:35.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 -- 172.21.15.118:0/3141645813 learned_addr learned my addr 172.21.15.118:0/3141645813 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:35.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547ad9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f547c158830 0x7f547c0c7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:44492/0 (socket says 172.21.15.118:44492) 2024-08-18T13:45:35.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f5479d9b700 1 --2- 172.21.15.118:0/3141645813 >> v2:172.21.15.178:3300/0 conn(0x7f547c14f7e0 0x7f547c0abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 -- 172.21.15.118:0/3141645813 >> v2:172.21.15.178:3300/0 conn(0x7f547c14f7e0 msgr2=0x7f547c0abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:35.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547959a700 1 -- 172.21.15.118:0/3141645813 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794819877 0 0) 0x7f547c0c1c80 con 0x7f547c158830 2024-08-18T13:45:35.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 --2- 172.21.15.118:0/3141645813 >> v2:172.21.15.178:3300/0 conn(0x7f547c14f7e0 0x7f547c0abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 -- 172.21.15.118:0/3141645813 >> v1:172.21.15.118:6789/0 conn(0x7f547c158830 legacy=0x7f547c0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 -- 172.21.15.118:0/3141645813 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0c0cd0 con 0x7f547c14f410 2024-08-18T13:45:35.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547a59c700 1 --2- 172.21.15.118:0/3141645813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0ab790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f547c0b19c0 tx=0x7f5464009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2313132b665332b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:35.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547959a700 1 -- 172.21.15.118:0/3141645813 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464009e70 con 0x7f547c14f410 2024-08-18T13:45:35.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547959a700 1 -- 172.21.15.118:0/3141645813 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400b8f0 con 0x7f547c14f410 2024-08-18T13:45:35.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.895+0000 7f547959a700 1 -- 172.21.15.118:0/3141645813 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400bc90 con 0x7f547c14f410 2024-08-18T13:45:35.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 -- 172.21.15.118:0/3141645813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 msgr2=0x7f547c0ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 --2- 172.21.15.118:0/3141645813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0ab790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f547c0b19c0 tx=0x7f5464009250 comp rx=0 tx=0).stop 2024-08-18T13:45:35.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 -- 172.21.15.118:0/3141645813 shutdown_connections 2024-08-18T13:45:35.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 --2- 172.21.15.118:0/3141645813 >> v2:172.21.15.178:3300/0 conn(0x7f547c14f7e0 0x7f547c0abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 --2- 172.21.15.118:0/3141645813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0ab790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 -- 172.21.15.118:0/3141645813 >> 172.21.15.118:0/3141645813 conn(0x7f547c0c8620 msgr2=0x7f547c0c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:35.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 -- 172.21.15.118:0/3141645813 shutdown_connections 2024-08-18T13:45:35.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 -- 172.21.15.118:0/3141645813 wait complete. 2024-08-18T13:45:35.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.899+0000 7f547b59e700 1 Processor -- start 2024-08-18T13:45:35.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 -- start start 2024-08-18T13:45:35.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f547c14f7e0 0x7f547c14b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547c158830 0x7f547c14b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c14ad40 con 0x7f547c14f410 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c1656b0 con 0x7f547c158830 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c165990 con 0x7f547c14f7e0 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37576/0 (socket says 172.21.15.118:37576) 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 -- 172.21.15.118:0/2525585947 learned_addr learned my addr 172.21.15.118:0/2525585947 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f5479d9b700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f547c14f7e0 0x7f547c14b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547ad9d700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547c158830 0x7f547c14b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 -- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f547c14f7e0 msgr2=0x7f547c14b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f547c14f7e0 0x7f547c14b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 -- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547c158830 msgr2=0x7f547c14b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:35.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547c158830 0x7f547c14b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:35.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 -- 172.21.15.118:0/2525585947 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640094f0 con 0x7f547c14f410 2024-08-18T13:45:35.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547a59c700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0af340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f54640094c0 tx=0x7f5464017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:35.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f54727fc700 1 -- 172.21.15.118:0/2525585947 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400bc70 con 0x7f547c14f410 2024-08-18T13:45:35.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f54727fc700 1 -- 172.21.15.118:0/2525585947 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5464024440 con 0x7f547c14f410 2024-08-18T13:45:35.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c165c70 con 0x7f547c14f410 2024-08-18T13:45:35.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f54727fc700 1 -- 172.21.15.118:0/2525585947 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464024800 con 0x7f547c14f410 2024-08-18T13:45:35.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.903+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c0c1de0 con 0x7f547c14f410 2024-08-18T13:45:35.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.907+0000 7f54727fc700 1 -- 172.21.15.118:0/2525585947 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f546401a420 con 0x7f547c14f410 2024-08-18T13:45:35.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.907+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c0713a0 con 0x7f547c14f410 2024-08-18T13:45:35.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.907+0000 7f54727fc700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f545c060890 0x7f545c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:35.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.907+0000 7f54727fc700 1 -- 172.21.15.118:0/2525585947 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f546407e9b0 con 0x7f547c14f410 2024-08-18T13:45:35.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.907+0000 7f5479d9b700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f545c060890 0x7f545c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:35.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.911+0000 7f5479d9b700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f545c060890 0x7f545c062d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f547c1563c0 tx=0x7f546c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:35.921 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:35.919+0000 7f54727fc700 1 -- 172.21.15.118:0/2525585947 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f546404e920 con 0x7f547c14f410 2024-08-18T13:45:36.123 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.119+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f547c06f380 con 0x7f547c14f410 2024-08-18T13:45:36.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.119+0000 7f54727fc700 1 -- 172.21.15.118:0/2525585947 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5366 (secure 0 0 0) 0x7f5464015090 con 0x7f547c14f410 2024-08-18T13:45:36.124 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:36.124 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:35.570401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:36.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f545c060890 msgr2=0x7f545c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:36.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f545c060890 0x7f545c062d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f547c1563c0 tx=0x7f546c00b040 comp rx=0 tx=0).stop 2024-08-18T13:45:36.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 msgr2=0x7f547c0af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:36.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0af340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f54640094c0 tx=0x7f5464017000 comp rx=0 tx=0).stop 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 shutdown_connections 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f545c060890 0x7f545c062d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f547c158830 0x7f547c14b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f547c14f7e0 0x7f547c14b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 --2- 172.21.15.118:0/2525585947 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f547c14f410 0x7f547c0af340 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 >> 172.21.15.118:0/2525585947 conn(0x7f547c0c8620 msgr2=0x7f547c158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 shutdown_connections 2024-08-18T13:45:36.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:36.123+0000 7f547b59e700 1 -- 172.21.15.118:0/2525585947 wait complete. 2024-08-18T13:45:36.128 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 21 2024-08-18T13:45:36.139 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 21, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 21, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5209": { "addr": "172.21.15.178:6835/679522662", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6832", "nonce": 679522662, "type": "v2" }, { "addr": "172.21.15.178:6835", "nonce": 679522662, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5209, "incarnation": 21, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:45:35.570401+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5209 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:45:38.140 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:38.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7d082c700 1 Processor -- start 2024-08-18T13:45:38.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7d082c700 1 -- start start 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7d082c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa7cc14b5a0 0x7fa7cc0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7d082c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc06f000 0x7fa7cc15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7d082c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7cc0c8730 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7d082c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc1520e0 con 0x7fa7cc14b5a0 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7d082c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0c2460 con 0x7fa7cc06f000 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa7cc14b5a0 0x7fa7cc0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa7cc14b5a0 0x7fa7cc0673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37586/0 (socket says 172.21.15.118:37586) 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 -- 172.21.15.118:0/2676155260 learned_addr learned my addr 172.21.15.118:0/2676155260 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cbfff700 1 --2- 172.21.15.118:0/2676155260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc06f000 0x7fa7cc15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:38.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 -- 172.21.15.118:0/2676155260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc06f000 msgr2=0x7fa7cc15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:38.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 --2- 172.21.15.118:0/2676155260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc06f000 0x7fa7cc15bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 -- 172.21.15.118:0/2676155260 >> v1:172.21.15.178:6789/0 conn(0x7fa7cc0c7fc0 legacy=0x7fa7cc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:38.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 -- 172.21.15.118:0/2676155260 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0c2720 con 0x7fa7cc14b5a0 2024-08-18T13:45:38.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7cb7fe700 1 --2- 172.21.15.118:0/2676155260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc14b5a0 0x7fa7cc0673d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00099e0 tx=0x7fa7c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df1dd086c07ab83 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:38.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7ca7fc700 1 -- 172.21.15.118:0/2676155260 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000b820 con 0x7fa7cc14b5a0 2024-08-18T13:45:38.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7ca7fc700 1 -- 172.21.15.118:0/2676155260 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c000b980 con 0x7fa7cc14b5a0 2024-08-18T13:45:38.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.251+0000 7fa7ca7fc700 1 -- 172.21.15.118:0/2676155260 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000bd20 con 0x7fa7cc14b5a0 2024-08-18T13:45:38.258 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 -- 172.21.15.118:0/2676155260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc14b5a0 msgr2=0x7fa7cc0673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:38.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2676155260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc14b5a0 0x7fa7cc0673d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00099e0 tx=0x7fa7c0009250 comp rx=0 tx=0).stop 2024-08-18T13:45:38.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 -- 172.21.15.118:0/2676155260 shutdown_connections 2024-08-18T13:45:38.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2676155260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc06f000 0x7fa7cc15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2676155260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc14b5a0 0x7fa7cc0673d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 -- 172.21.15.118:0/2676155260 >> 172.21.15.118:0/2676155260 conn(0x7fa7cc06c020 msgr2=0x7fa7cc0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:38.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 -- 172.21.15.118:0/2676155260 shutdown_connections 2024-08-18T13:45:38.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.255+0000 7fa7d082c700 1 -- 172.21.15.118:0/2676155260 wait complete. 2024-08-18T13:45:38.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 Processor -- start 2024-08-18T13:45:38.261 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 -- start start 2024-08-18T13:45:38.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7cc06f000 0x7fa7cc0bfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc0c7fc0 0x7fa7cc150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc14b5a0 0x7fa7cc153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc068870 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0bf7f0 con 0x7fa7cc06f000 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc152490 con 0x7fa7cc14b5a0 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc0c7fc0 0x7fa7cc150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc0c7fc0 0x7fa7cc150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37592/0 (socket says 172.21.15.118:37592) 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 -- 172.21.15.118:0/2452755084 learned_addr learned my addr 172.21.15.118:0/2452755084 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:38.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7cb7fe700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7cc06f000 0x7fa7cc0bfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7cbfff700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc14b5a0 0x7fa7cc153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 -- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc14b5a0 msgr2=0x7fa7cc153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc14b5a0 0x7fa7cc153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 -- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7cc06f000 msgr2=0x7fa7cc0bfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7cc06f000 0x7fa7cc0bfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 -- 172.21.15.118:0/2452755084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00094f0 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7caffd700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc0c7fc0 0x7fa7cc150300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c5c0 tx=0x7fa7bc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7b37fe700 1 -- 172.21.15.118:0/2452755084 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc001120 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7b37fe700 1 -- 172.21.15.118:0/2452755084 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc011590 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc068aa0 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.259+0000 7fa7b37fe700 1 -- 172.21.15.118:0/2452755084 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc011950 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.267 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.263+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc0bf480 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.263+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc0bf2e0 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.263+0000 7fa7b37fe700 1 -- 172.21.15.118:0/2452755084 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa7bc011d00 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.263+0000 7fa7b37fe700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7b8060890 0x7fa7b8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:38.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.263+0000 7fa7b37fe700 1 -- 172.21.15.118:0/2452755084 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fa7bc07cac0 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.263+0000 7fa7cb7fe700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7b8060890 0x7fa7b8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:38.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.263+0000 7fa7cb7fe700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7b8060890 0x7fa7b8062d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00099e0 tx=0x7fa7c0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:38.275 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.271+0000 7fa7b37fe700 1 -- 172.21.15.118:0/2452755084 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa7cc0bf2e0 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.471+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7cc0bf2e0 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7b37fe700 1 -- 172.21.15.118:0/2452755084 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5366 (secure 0 0 0) 0x7fa7cc0bf2e0 con 0x7fa7cc0c7fc0 2024-08-18T13:45:38.477 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:38.478 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:35.570401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7b8060890 msgr2=0x7fa7b8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7b8060890 0x7fa7b8062d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00099e0 tx=0x7fa7c0017000 comp rx=0 tx=0).stop 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc0c7fc0 msgr2=0x7fa7cc150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc0c7fc0 0x7fa7cc150300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c5c0 tx=0x7fa7bc007f30 comp rx=0 tx=0).stop 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 shutdown_connections 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa7b8060890 0x7fa7b8062d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7cc14b5a0 0x7fa7cc153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7cc0c7fc0 0x7fa7cc150300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 --2- 172.21.15.118:0/2452755084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7cc06f000 0x7fa7cc0bfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 >> 172.21.15.118:0/2452755084 conn(0x7fa7cc06c020 msgr2=0x7fa7cc06e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:38.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 shutdown_connections 2024-08-18T13:45:38.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:38.475+0000 7fa7d082c700 1 -- 172.21.15.118:0/2452755084 wait complete. 2024-08-18T13:45:38.481 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 21 2024-08-18T13:45:38.491 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:45:40.492 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:40.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.599+0000 7f680f59e700 1 Processor -- start 2024-08-18T13:45:40.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.599+0000 7f680f59e700 1 -- start start 2024-08-18T13:45:40.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f681014f7e0 0x7f68100c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:40.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f68100abea0 0x7f68100c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:40.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68100c1c80 con 0x7f681014f410 2024-08-18T13:45:40.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100c18e0 con 0x7f68100abea0 2024-08-18T13:45:40.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68100c09f0 con 0x7f681014f7e0 2024-08-18T13:45:40.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f68100abea0 0x7f68100c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:40.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f68100abea0 0x7f68100c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37600/0 (socket says 172.21.15.118:37600) 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 -- 172.21.15.118:0/1829502115 learned_addr learned my addr 172.21.15.118:0/1829502115 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f6807fff700 1 --2- 172.21.15.118:0/1829502115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f681014f7e0 0x7f68100c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 -- 172.21.15.118:0/1829502115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f681014f7e0 msgr2=0x7f68100c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 --2- 172.21.15.118:0/1829502115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f681014f7e0 0x7f68100c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 -- 172.21.15.118:0/1829502115 >> v1:172.21.15.178:6789/0 conn(0x7f681014f410 legacy=0x7f68100ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 -- 172.21.15.118:0/1829502115 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100c0cd0 con 0x7f68100abea0 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680dd9b700 1 -- 172.21.15.118:0/1829502115 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650249641 0 0) 0x7f68100c1c80 con 0x7f681014f410 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680ed9d700 1 --2- 172.21.15.118:0/1829502115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68100abea0 0x7f68100c7a70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6808009f20 tx=0x7f6808009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf286d65f1ada2c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680dd9b700 1 -- 172.21.15.118:0/1829502115 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800e070 con 0x7f68100abea0 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680dd9b700 1 -- 172.21.15.118:0/1829502115 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680800ba60 con 0x7f68100abea0 2024-08-18T13:45:40.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680dd9b700 1 -- 172.21.15.118:0/1829502115 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080092e0 con 0x7f68100abea0 2024-08-18T13:45:40.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- 172.21.15.118:0/1829502115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68100abea0 msgr2=0x7f68100c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:40.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 --2- 172.21.15.118:0/1829502115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68100abea0 0x7f68100c7a70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6808009f20 tx=0x7f6808009840 comp rx=0 tx=0).stop 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- 172.21.15.118:0/1829502115 shutdown_connections 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 --2- 172.21.15.118:0/1829502115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68100abea0 0x7f68100c7a70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 --2- 172.21.15.118:0/1829502115 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f681014f7e0 0x7f68100c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- 172.21.15.118:0/1829502115 >> 172.21.15.118:0/1829502115 conn(0x7f68100c8620 msgr2=0x7f68100c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- 172.21.15.118:0/1829502115 shutdown_connections 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- 172.21.15.118:0/1829502115 wait complete. 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 Processor -- start 2024-08-18T13:45:40.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.603+0000 7f680f59e700 1 -- start start 2024-08-18T13:45:40.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68100abea0 0x7f681014a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:40.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f681014f410 0x7f6810155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:40.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f681014f7e0 0x7f6810164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:40.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681015a7d0 con 0x7f681014f7e0 2024-08-18T13:45:40.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810159ce0 con 0x7f681014f410 2024-08-18T13:45:40.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681015a340 con 0x7f68100abea0 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f681014f7e0 0x7f6810164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f681014f7e0 0x7f6810164eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37608/0 (socket says 172.21.15.118:37608) 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 -- 172.21.15.118:0/1561198929 learned_addr learned my addr 172.21.15.118:0/1561198929 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f6807fff700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f681014f410 0x7f6810155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680e59c700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68100abea0 0x7f681014a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 -- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68100abea0 msgr2=0x7f681014a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68100abea0 0x7f681014a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 -- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f681014f410 msgr2=0x7f6810155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f681014f410 0x7f6810155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 -- 172.21.15.118:0/1561198929 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808009ae0 con 0x7f681014f7e0 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680ed9d700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f681014f7e0 0x7f6810164eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68100896a0 tx=0x7f6808018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f68067fc700 1 -- 172.21.15.118:0/1561198929 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080032f0 con 0x7f681014f7e0 2024-08-18T13:45:40.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f68067fc700 1 -- 172.21.15.118:0/1561198929 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808008ce0 con 0x7f681014f7e0 2024-08-18T13:45:40.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6810159a00 con 0x7f681014f7e0 2024-08-18T13:45:40.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f68067fc700 1 -- 172.21.15.118:0/1561198929 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080266b0 con 0x7f681014f7e0 2024-08-18T13:45:40.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6810148e10 con 0x7f681014f7e0 2024-08-18T13:45:40.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68100713a0 con 0x7f681014f7e0 2024-08-18T13:45:40.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.607+0000 7f68067fc700 1 -- 172.21.15.118:0/1561198929 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6808003040 con 0x7f681014f7e0 2024-08-18T13:45:40.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.611+0000 7f68067fc700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f67f0060860 0x7f67f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:40.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.611+0000 7f68067fc700 1 -- 172.21.15.118:0/1561198929 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f680807ee70 con 0x7f681014f7e0 2024-08-18T13:45:40.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.611+0000 7f680e59c700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f67f0060860 0x7f67f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:40.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.611+0000 7f680e59c700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f67f0060860 0x7f67f0062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68000011e0 tx=0x7f6800007ce0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:40.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.615+0000 7f68067fc700 1 -- 172.21.15.118:0/1561198929 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f680804ee00 con 0x7f681014f7e0 2024-08-18T13:45:40.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.815+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6810158050 con 0x7f681014f7e0 2024-08-18T13:45:40.819 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.815+0000 7f68067fc700 1 -- 172.21.15.118:0/1561198929 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5366 (secure 0 0 0) 0x7f6808052420 con 0x7f681014f7e0 2024-08-18T13:45:40.819 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:40.820 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:35.570401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:40.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f67f0060860 msgr2=0x7f67f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:40.823 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f67f0060860 0x7f67f0062d10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68000011e0 tx=0x7f6800007ce0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f681014f7e0 msgr2=0x7f6810164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f681014f7e0 0x7f6810164eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68100896a0 tx=0x7f6808018000 comp rx=0 tx=0).stop 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 shutdown_connections 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f67f0060860 0x7f67f0062d10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f681014f7e0 0x7f6810164eb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f681014f410 0x7f6810155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 --2- 172.21.15.118:0/1561198929 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68100abea0 0x7f681014a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 >> 172.21.15.118:0/1561198929 conn(0x7f68100c8620 msgr2=0x7f68100c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 shutdown_connections 2024-08-18T13:45:40.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:40.819+0000 7f680f59e700 1 -- 172.21.15.118:0/1561198929 wait complete. 2024-08-18T13:45:40.825 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 21 2024-08-18T13:45:40.839 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:45:42.840 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:42.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 Processor -- start 2024-08-18T13:45:42.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- start start 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f482806b620 0x7f482806b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482806bf30 con 0x7f48280c4030 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4828068ac0 con 0x7f48280c2630 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4828068580 con 0x7f482806b620 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f48277fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f482806b620 0x7f482806b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f48277fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f482806b620 0x7f482806b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37620/0 (socket says 172.21.15.118:37620) 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f481effd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f48280c4030 0x7f4828152c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:44500/0 (socket says 172.21.15.118:44500) 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f48277fe700 1 -- 172.21.15.118:0/182002190 learned_addr learned my addr 172.21.15.118:0/182002190 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f48277fe700 1 -- 172.21.15.118:0/182002190 >> v1:172.21.15.178:6789/0 conn(0x7f48280c2630 legacy=0x7f482814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:42.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f4826ffd700 1 -- 172.21.15.118:0/182002190 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036481982 0 0) 0x7f482806bf30 con 0x7f48280c4030 2024-08-18T13:45:42.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f48277fe700 1 -- 172.21.15.118:0/182002190 >> v1:172.21.15.118:6789/0 conn(0x7f48280c4030 legacy=0x7f4828152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:42.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f48277fe700 1 -- 172.21.15.118:0/182002190 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828068820 con 0x7f482806b620 2024-08-18T13:45:42.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f48277fe700 1 --2- 172.21.15.118:0/182002190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f482806b620 0x7f482806b9f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48180099b0 tx=0x7f4818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9604388586270c83 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:42.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f4826ffd700 1 -- 172.21.15.118:0/182002190 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800b860 con 0x7f482806b620 2024-08-18T13:45:42.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f4826ffd700 1 -- 172.21.15.118:0/182002190 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481800b9c0 con 0x7f482806b620 2024-08-18T13:45:42.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f4826ffd700 1 -- 172.21.15.118:0/182002190 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800bd60 con 0x7f482806b620 2024-08-18T13:45:42.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- 172.21.15.118:0/182002190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f482806b620 msgr2=0x7f482806b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:42.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 --2- 172.21.15.118:0/182002190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f482806b620 0x7f482806b9f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f48180099b0 tx=0x7f4818009250 comp rx=0 tx=0).stop 2024-08-18T13:45:42.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- 172.21.15.118:0/182002190 shutdown_connections 2024-08-18T13:45:42.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 --2- 172.21.15.118:0/182002190 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f482806b620 0x7f482806b9f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:42.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- 172.21.15.118:0/182002190 >> 172.21.15.118:0/182002190 conn(0x7f48280c7c40 msgr2=0x7f48280c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:42.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- 172.21.15.118:0/182002190 shutdown_connections 2024-08-18T13:45:42.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.951+0000 7f482dc9c700 1 -- 172.21.15.118:0/182002190 wait complete. 2024-08-18T13:45:42.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 Processor -- start 2024-08-18T13:45:42.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 -- start start 2024-08-18T13:45:42.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f482806b620 0x7f4828159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:42.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48280c2630 0x7f4828154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:42.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48280c4030 0x7f48281548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:42.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828165340 con 0x7f48280c2630 2024-08-18T13:45:42.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48281688c0 con 0x7f482806b620 2024-08-18T13:45:42.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48280be1f0 con 0x7f48280c4030 2024-08-18T13:45:42.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48280c2630 0x7f4828154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48280c2630 0x7f4828154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37636/0 (socket says 172.21.15.118:37636) 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 -- 172.21.15.118:0/1271916921 learned_addr learned my addr 172.21.15.118:0/1271916921 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f48277fe700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f482806b620 0x7f4828159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f4827fff700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48280c4030 0x7f48281548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 -- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48280c4030 msgr2=0x7f48281548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48280c4030 0x7f48281548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 -- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f482806b620 msgr2=0x7f4828159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f482806b620 0x7f4828159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 -- 172.21.15.118:0/1271916921 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180094f0 con 0x7f48280c2630 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481effd700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48280c2630 0x7f4828154370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f481000c810 tx=0x7f4810007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481ffff700 1 -- 172.21.15.118:0/1271916921 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810005b40 con 0x7f48280c2630 2024-08-18T13:45:42.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481ffff700 1 -- 172.21.15.118:0/1271916921 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4810005cc0 con 0x7f48280c2630 2024-08-18T13:45:42.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4828165530 con 0x7f48280c2630 2024-08-18T13:45:42.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f481ffff700 1 -- 172.21.15.118:0/1271916921 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810007320 con 0x7f48280c2630 2024-08-18T13:45:42.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.955+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4828168db0 con 0x7f48280c2630 2024-08-18T13:45:42.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.959+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48280642f0 con 0x7f48280c2630 2024-08-18T13:45:42.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.959+0000 7f481ffff700 1 -- 172.21.15.118:0/1271916921 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f481001b3b0 con 0x7f48280c2630 2024-08-18T13:45:42.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.959+0000 7f481ffff700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4808060860 0x7f4808062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:42.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.959+0000 7f481ffff700 1 -- 172.21.15.118:0/1271916921 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f481007ca40 con 0x7f48280c2630 2024-08-18T13:45:42.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.959+0000 7f48277fe700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4808060860 0x7f4808062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:42.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.963+0000 7f48277fe700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4808060860 0x7f4808062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48180099b0 tx=0x7f4818017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:42.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:42.963+0000 7f481ffff700 1 -- 172.21.15.118:0/1271916921 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f481004c9b0 con 0x7f48280c2630 2024-08-18T13:45:43.130 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:45:43.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:45:43.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48280c2310 con 0x7f48280c2630 2024-08-18T13:45:43.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f481ffff700 1 -- 172.21.15.118:0/1271916921 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5366 (secure 0 0 0) 0x7f4810017090 con 0x7f48280c2630 2024-08-18T13:45:43.166 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:43.166 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:35.570401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:43.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4808060860 msgr2=0x7f4808062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:43.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4808060860 0x7f4808062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48180099b0 tx=0x7f4818017000 comp rx=0 tx=0).stop 2024-08-18T13:45:43.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48280c2630 msgr2=0x7f4828154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:43.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48280c2630 0x7f4828154370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f481000c810 tx=0x7f4810007d90 comp rx=0 tx=0).stop 2024-08-18T13:45:43.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 shutdown_connections 2024-08-18T13:45:43.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4808060860 0x7f4808062d10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:43.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48280c4030 0x7f48281548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:43.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48280c2630 0x7f4828154370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:43.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 --2- 172.21.15.118:0/1271916921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f482806b620 0x7f4828159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:43.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 >> 172.21.15.118:0/1271916921 conn(0x7f48280c7c40 msgr2=0x7f4828150060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:43.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 shutdown_connections 2024-08-18T13:45:43.170 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:43.163+0000 7f482dc9c700 1 -- 172.21.15.118:0/1271916921 wait complete. 2024-08-18T13:45:43.171 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 21 2024-08-18T13:45:43.180 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:45:45.181 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:45.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.291+0000 7f1c4f0d4700 1 Processor -- start 2024-08-18T13:45:45.296 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.291+0000 7f1c4f0d4700 1 -- start start 2024-08-18T13:45:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c480c3680 0x7f1c480673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1c480c3e70 0x7f1c4814d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4806cc90 con 0x7f1c480c3a50 2024-08-18T13:45:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c480c83a0 con 0x7f1c480c3e70 2024-08-18T13:45:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c48153090 con 0x7f1c480c3680 2024-08-18T13:45:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1c480c3e70 0x7f1c4814d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:45.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1c480c3e70 0x7f1c4814d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58132/0 (socket says 172.21.15.118:58132) 2024-08-18T13:45:45.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 -- 172.21.15.118:0/808493100 learned_addr learned my addr 172.21.15.118:0/808493100 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:45.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e0d2700 1 --2- 172.21.15.118:0/808493100 >> v2:172.21.15.178:3300/0 conn(0x7f1c480c3680 0x7f1c480673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:45.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 -- 172.21.15.118:0/808493100 >> v2:172.21.15.178:3300/0 conn(0x7f1c480c3680 msgr2=0x7f1c480673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:45.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 --2- 172.21.15.118:0/808493100 >> v2:172.21.15.178:3300/0 conn(0x7f1c480c3680 0x7f1c480673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 -- 172.21.15.118:0/808493100 >> v1:172.21.15.178:6789/0 conn(0x7f1c480c3a50 legacy=0x7f1c4814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:45.299 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 -- 172.21.15.118:0/808493100 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c480c8660 con 0x7f1c480c3e70 2024-08-18T13:45:45.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4e8d3700 1 --2- 172.21.15.118:0/808493100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3e70 0x7f1c4814d410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c4806d3b0 tx=0x7f1c40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5044706a39835f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:45.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4d0d0700 1 -- 172.21.15.118:0/808493100 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000e070 con 0x7f1c480c3e70 2024-08-18T13:45:45.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4d0d0700 1 -- 172.21.15.118:0/808493100 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4000ba50 con 0x7f1c480c3e70 2024-08-18T13:45:45.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4d0d0700 1 -- 172.21.15.118:0/808493100 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400092e0 con 0x7f1c480c3e70 2024-08-18T13:45:45.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/808493100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3e70 msgr2=0x7f1c4814d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:45.302 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/808493100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3e70 0x7f1c4814d410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c4806d3b0 tx=0x7f1c40009840 comp rx=0 tx=0).stop 2024-08-18T13:45:45.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/808493100 shutdown_connections 2024-08-18T13:45:45.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/808493100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3e70 0x7f1c4814d410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/808493100 >> v2:172.21.15.178:3300/0 conn(0x7f1c480c3680 0x7f1c480673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/808493100 >> 172.21.15.118:0/808493100 conn(0x7f1c4806c020 msgr2=0x7f1c480c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:45.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/808493100 shutdown_connections 2024-08-18T13:45:45.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.295+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/808493100 wait complete. 2024-08-18T13:45:45.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 Processor -- start 2024-08-18T13:45:45.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 -- start start 2024-08-18T13:45:45.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3680 0x7f1c48166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:45.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c480c3a50 0x7f1c480c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:45.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c480c3e70 0x7f1c481450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:45.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4814b010 con 0x7f1c480c3680 2024-08-18T13:45:45.308 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481663a0 con 0x7f1c480c3e70 2024-08-18T13:45:45.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4f0d4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c48145e20 con 0x7f1c480c3a50 2024-08-18T13:45:45.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3680 0x7f1c48166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:45.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3680 0x7f1c48166830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58134/0 (socket says 172.21.15.118:58134) 2024-08-18T13:45:45.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 -- 172.21.15.118:0/1636184974 learned_addr learned my addr 172.21.15.118:0/1636184974 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:45.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e8d3700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c480c3e70 0x7f1c481450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:45.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4d8d1700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c480c3a50 0x7f1c480c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:45.309 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 -- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c480c3a50 msgr2=0x7f1c480c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:45.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c480c3a50 0x7f1c480c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 -- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c480c3e70 msgr2=0x7f1c481450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:45.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c480c3e70 0x7f1c481450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.299+0000 7f1c4e0d2700 1 -- 172.21.15.118:0/1636184974 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c40009ae0 con 0x7f1c480c3680 2024-08-18T13:45:45.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c4e8d3700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c480c3e70 0x7f1c481450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:45:45.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c4e0d2700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3680 0x7f1c48166830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800d890 tx=0x7f1c38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:45.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c3e7fc700 1 -- 172.21.15.118:0/1636184974 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38009610 con 0x7f1c480c3680 2024-08-18T13:45:45.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c3e7fc700 1 -- 172.21.15.118:0/1636184974 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38009770 con 0x7f1c480c3680 2024-08-18T13:45:45.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c3e7fc700 1 -- 172.21.15.118:0/1636184974 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38009b30 con 0x7f1c480c3680 2024-08-18T13:45:45.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c481460c0 con 0x7f1c480c3680 2024-08-18T13:45:45.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4814ad30 con 0x7f1c480c3680 2024-08-18T13:45:45.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c480713a0 con 0x7f1c480c3680 2024-08-18T13:45:45.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c3e7fc700 1 -- 172.21.15.118:0/1636184974 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1c3801e460 con 0x7f1c480c3680 2024-08-18T13:45:45.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c3e7fc700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1c2c060890 0x7f1c2c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:45.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c4d8d1700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1c2c060890 0x7f1c2c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:45.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.303+0000 7f1c3e7fc700 1 -- 172.21.15.118:0/1636184974 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f1c3807d730 con 0x7f1c480c3680 2024-08-18T13:45:45.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.307+0000 7f1c4d8d1700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1c2c060890 0x7f1c2c062d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c4814cd50 tx=0x7f1c44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:45.314 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.311+0000 7f1c3e7fc700 1 -- 172.21.15.118:0/1636184974 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1c3804d620 con 0x7f1c480c3680 2024-08-18T13:45:45.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.511+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c4814caa0 con 0x7f1c480c3680 2024-08-18T13:45:45.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.511+0000 7f1c3e7fc700 1 -- 172.21.15.118:0/1636184974 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f1c38050c40 con 0x7f1c480c3680 2024-08-18T13:45:45.515 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:45.515 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:43.420167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:45.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1c2c060890 msgr2=0x7f1c2c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:45.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1c2c060890 0x7f1c2c062d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c4814cd50 tx=0x7f1c44009040 comp rx=0 tx=0).stop 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3680 msgr2=0x7f1c48166830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3680 0x7f1c48166830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800d890 tx=0x7f1c38008040 comp rx=0 tx=0).stop 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 shutdown_connections 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1c2c060890 0x7f1c2c062d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c480c3e70 0x7f1c481450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c480c3a50 0x7f1c480c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 --2- 172.21.15.118:0/1636184974 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c480c3680 0x7f1c48166830 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 >> 172.21.15.118:0/1636184974 conn(0x7f1c4806c020 msgr2=0x7f1c48151f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:45.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 shutdown_connections 2024-08-18T13:45:45.519 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:45.515+0000 7f1c4f0d4700 1 -- 172.21.15.118:0/1636184974 wait complete. 2024-08-18T13:45:45.519 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 22 2024-08-18T13:45:45.529 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:45:47.530 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:47.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f359e700 1 Processor -- start 2024-08-18T13:45:47.647 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f359e700 1 -- start start 2024-08-18T13:45:47.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f359e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39f414f410 0x7f39f40ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:47.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f40c1c80 con 0x7f39f4158830 2024-08-18T13:45:47.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f359e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f40c18e0 con 0x7f39f414f7e0 2024-08-18T13:45:47.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f359e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f40c09f0 con 0x7f39f414f410 2024-08-18T13:45:47.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f2d9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f39f4158830 0x7f39f415fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:33342/0 (socket says 172.21.15.118:33342) 2024-08-18T13:45:47.648 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f2d9d700 1 -- 172.21.15.118:0/2938368728 learned_addr learned my addr 172.21.15.118:0/2938368728 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:45:47.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f259c700 1 --2- 172.21.15.118:0/2938368728 >> v2:172.21.15.178:3300/0 conn(0x7f39f414f410 0x7f39f40ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:47.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.643+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890906589 0 0) 0x7f39f40c1c80 con 0x7f39f4158830 2024-08-18T13:45:47.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d4003570 con 0x7f39f4158830 2024-08-18T13:45:47.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750773773 0 0) 0x7f39f40c18e0 con 0x7f39f414f7e0 2024-08-18T13:45:47.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39f40c1c80 con 0x7f39f414f7e0 2024-08-18T13:45:47.650 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3839112409 0 0) 0x7f39d4003570 con 0x7f39f4158830 2024-08-18T13:45:47.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f259c700 1 -- 172.21.15.118:0/2938368728 >> v1:172.21.15.178:6789/0 conn(0x7f39f414f7e0 legacy=0x7f39f40c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:47.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f259c700 1 -- 172.21.15.118:0/2938368728 >> v1:172.21.15.118:6789/0 conn(0x7f39f4158830 legacy=0x7f39f415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:47.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f259c700 1 -- 172.21.15.118:0/2938368728 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40c0cd0 con 0x7f39f414f410 2024-08-18T13:45:47.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f259c700 1 --2- 172.21.15.118:0/2938368728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 0x7f39f40ab790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0098e0 tx=0x7f39ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c542c89531f594e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:47.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0017f0 con 0x7f39f414f410 2024-08-18T13:45:47.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec011420 con 0x7f39f414f410 2024-08-18T13:45:47.651 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f159a700 1 -- 172.21.15.118:0/2938368728 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0117c0 con 0x7f39f414f410 2024-08-18T13:45:47.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f359e700 1 -- 172.21.15.118:0/2938368728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 msgr2=0x7f39f40ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:47.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f359e700 1 --2- 172.21.15.118:0/2938368728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 0x7f39f40ab790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0098e0 tx=0x7f39ec0091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f359e700 1 -- 172.21.15.118:0/2938368728 shutdown_connections 2024-08-18T13:45:47.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f359e700 1 --2- 172.21.15.118:0/2938368728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 0x7f39f40ab790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.652 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f359e700 1 -- 172.21.15.118:0/2938368728 >> 172.21.15.118:0/2938368728 conn(0x7f39f40c8620 msgr2=0x7f39f40c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:47.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f359e700 1 -- 172.21.15.118:0/2938368728 shutdown_connections 2024-08-18T13:45:47.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.647+0000 7f39f359e700 1 -- 172.21.15.118:0/2938368728 wait complete. 2024-08-18T13:45:47.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 Processor -- start 2024-08-18T13:45:47.653 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 -- start start 2024-08-18T13:45:47.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 0x7f39f4168a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:47.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39f414f7e0 0x7f39f414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:47.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39f4158830 0x7f39f414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:47.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f41696e0 con 0x7f39f4158830 2024-08-18T13:45:47.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f40694d0 con 0x7f39f414f410 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4069d20 con 0x7f39f414f7e0 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39f4158830 0x7f39f414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39f4158830 0x7f39f414da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58148/0 (socket says 172.21.15.118:58148) 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 -- 172.21.15.118:0/818161391 learned_addr learned my addr 172.21.15.118:0/818161391 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f259c700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 0x7f39f4168a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f1d9b700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39f414f7e0 0x7f39f414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 -- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39f414f7e0 msgr2=0x7f39f414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39f414f7e0 0x7f39f414d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 -- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 msgr2=0x7f39f4168a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 0x7f39f4168a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 -- 172.21.15.118:0/818161391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec009420 con 0x7f39f4158830 2024-08-18T13:45:47.656 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f2d9d700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39f4158830 0x7f39f414da30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00cc00 tx=0x7f39dc007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:47.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39ea7fc700 1 -- 172.21.15.118:0/818161391 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc004dc0 con 0x7f39f4158830 2024-08-18T13:45:47.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39ea7fc700 1 -- 172.21.15.118:0/818161391 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc007150 con 0x7f39f4158830 2024-08-18T13:45:47.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f4157660 con 0x7f39f4158830 2024-08-18T13:45:47.657 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39ea7fc700 1 -- 172.21.15.118:0/818161391 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc007510 con 0x7f39f4158830 2024-08-18T13:45:47.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.651+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f4157380 con 0x7f39f4158830 2024-08-18T13:45:47.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.655+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f40713a0 con 0x7f39f4158830 2024-08-18T13:45:47.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.655+0000 7f39ea7fc700 1 -- 172.21.15.118:0/818161391 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f39dc00a4f0 con 0x7f39f4158830 2024-08-18T13:45:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.655+0000 7f39ea7fc700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f39d8060860 0x7f39d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.655+0000 7f39ea7fc700 1 -- 172.21.15.118:0/818161391 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f39dc07c6e0 con 0x7f39f4158830 2024-08-18T13:45:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.655+0000 7f39f259c700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f39d8060860 0x7f39d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.655+0000 7f39f259c700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f39d8060860 0x7f39d8062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0093f0 tx=0x7f39ec016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:47.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.663+0000 7f39ea7fc700 1 -- 172.21.15.118:0/818161391 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f39dc04c6f0 con 0x7f39f4158830 2024-08-18T13:45:47.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.863+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39f4069e60 con 0x7f39f4158830 2024-08-18T13:45:47.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39ea7fc700 1 -- 172.21.15.118:0/818161391 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f39dc04fd10 con 0x7f39f4158830 2024-08-18T13:45:47.869 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:47.869 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:43.420167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:47.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f39d8060860 msgr2=0x7f39d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:47.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f39d8060860 0x7f39d8062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0093f0 tx=0x7f39ec016000 comp rx=0 tx=0).stop 2024-08-18T13:45:47.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39f4158830 msgr2=0x7f39f414da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:47.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39f4158830 0x7f39f414da30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00cc00 tx=0x7f39dc007d20 comp rx=0 tx=0).stop 2024-08-18T13:45:47.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 shutdown_connections 2024-08-18T13:45:47.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f39d8060860 0x7f39d8062d10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f39f4158830 0x7f39f414da30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39f414f7e0 0x7f39f414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 --2- 172.21.15.118:0/818161391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f414f410 0x7f39f4168a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:47.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 >> 172.21.15.118:0/818161391 conn(0x7f39f40c8620 msgr2=0x7f39f4158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:47.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 shutdown_connections 2024-08-18T13:45:47.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:47.867+0000 7f39f359e700 1 -- 172.21.15.118:0/818161391 wait complete. 2024-08-18T13:45:47.873 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 22 2024-08-18T13:45:47.884 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5209": { "addr": "172.21.15.178:6835/679522662", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6832", "nonce": 679522662, "type": "v2" }, { "addr": "172.21.15.178:6835", "nonce": 679522662, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5209, "incarnation": 21, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:reconnect", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:45:43.420167+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5209 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:45:49.885 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:50.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:49.998+0000 7f49961f9700 1 Processor -- start 2024-08-18T13:45:50.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:49.998+0000 7f49961f9700 1 -- start start 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:49.998+0000 7f49961f9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f499014b940 0x7f499006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:49.998+0000 7f49961f9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f49901523d0 0x7f499006ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:49.998+0000 7f49961f9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499006de00 con 0x7f49900be5a0 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:49.998+0000 7f49961f9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499006fac0 con 0x7f499014b940 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:49.998+0000 7f49961f9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4990151870 con 0x7f49901523d0 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f499014b940 0x7f499006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f499014b940 0x7f499006e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58150/0 (socket says 172.21.15.118:58150) 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 -- 172.21.15.118:0/451375101 learned_addr learned my addr 172.21.15.118:0/451375101 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49959f8700 1 --2- 172.21.15.118:0/451375101 >> v2:172.21.15.178:3300/0 conn(0x7f49901523d0 0x7f499006ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:50.008 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 -- 172.21.15.118:0/451375101 >> v2:172.21.15.178:3300/0 conn(0x7f49901523d0 msgr2=0x7f499006ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 --2- 172.21.15.118:0/451375101 >> v2:172.21.15.178:3300/0 conn(0x7f49901523d0 0x7f499006ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 -- 172.21.15.118:0/451375101 >> v1:172.21.15.178:6789/0 conn(0x7f49900be5a0 legacy=0x7f4990151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 -- 172.21.15.118:0/451375101 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900c2da0 con 0x7f499014b940 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49959f8700 1 --2- 172.21.15.118:0/451375101 >> v2:172.21.15.178:3300/0 conn(0x7f49901523d0 0x7f499006ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49951f7700 1 --2- 172.21.15.118:0/451375101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499014b940 0x7f499006e4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f49900701b0 tx=0x7f498c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4051a2e95df79aec server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f4987fff700 1 -- 172.21.15.118:0/451375101 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c00b820 con 0x7f499014b940 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f4987fff700 1 -- 172.21.15.118:0/451375101 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c00b980 con 0x7f499014b940 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f4987fff700 1 -- 172.21.15.118:0/451375101 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c00bd20 con 0x7f499014b940 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 -- 172.21.15.118:0/451375101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499014b940 msgr2=0x7f499006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 --2- 172.21.15.118:0/451375101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499014b940 0x7f499006e4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f49900701b0 tx=0x7f498c009250 comp rx=0 tx=0).stop 2024-08-18T13:45:50.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 -- 172.21.15.118:0/451375101 shutdown_connections 2024-08-18T13:45:50.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 --2- 172.21.15.118:0/451375101 >> v2:172.21.15.178:3300/0 conn(0x7f49901523d0 0x7f499006ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 --2- 172.21.15.118:0/451375101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f499014b940 0x7f499006e4c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 -- 172.21.15.118:0/451375101 >> 172.21.15.118:0/451375101 conn(0x7f499006c020 msgr2=0x7f49900c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:50.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 -- 172.21.15.118:0/451375101 shutdown_connections 2024-08-18T13:45:50.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 -- 172.21.15.118:0/451375101 wait complete. 2024-08-18T13:45:50.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 Processor -- start 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.002+0000 7f49961f9700 1 -- start start 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49900be5a0 0x7f4990149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f499014b940 0x7f49900c1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49901523d0 0x7f49900c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c10e0 con 0x7f49901523d0 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900bfec0 con 0x7f499014b940 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c0180 con 0x7f49900be5a0 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49901523d0 0x7f49900c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49901523d0 0x7f49900c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58152/0 (socket says 172.21.15.118:58152) 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 -- 172.21.15.118:0/237747959 learned_addr learned my addr 172.21.15.118:0/237747959 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49951f7700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49900be5a0 0x7f4990149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49949f6700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f499014b940 0x7f49900c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:50.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 -- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49900be5a0 msgr2=0x7f4990149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49900be5a0 0x7f4990149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 -- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f499014b940 msgr2=0x7f49900c1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f499014b940 0x7f49900c1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 -- 172.21.15.118:0/237747959 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0094f0 con 0x7f49901523d0 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49951f7700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49900be5a0 0x7f4990149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49959f8700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49901523d0 0x7f49900c5f90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f498800a380 tx=0x7f4988007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49857fa700 1 -- 172.21.15.118:0/237747959 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498800aea0 con 0x7f49901523d0 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49857fa700 1 -- 172.21.15.118:0/237747959 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498800d650 con 0x7f49901523d0 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900c07b0 con 0x7f49901523d0 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49857fa700 1 -- 172.21.15.118:0/237747959 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498800da10 con 0x7f49901523d0 2024-08-18T13:45:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.006+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49900c0d70 con 0x7f49901523d0 2024-08-18T13:45:50.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.010+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49900c1290 con 0x7f49901523d0 2024-08-18T13:45:50.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.010+0000 7f49857fa700 1 -- 172.21.15.118:0/237747959 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f498801e3b0 con 0x7f49901523d0 2024-08-18T13:45:50.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.010+0000 7f49857fa700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4974060890 0x7f4974062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:50.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.010+0000 7f49857fa700 1 -- 172.21.15.118:0/237747959 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f498807dad0 con 0x7f49901523d0 2024-08-18T13:45:50.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.010+0000 7f49951f7700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4974060890 0x7f4974062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:50.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.010+0000 7f49951f7700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4974060890 0x7f4974062d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49900701b0 tx=0x7f498c016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:50.024 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.018+0000 7f49857fa700 1 -- 172.21.15.118:0/237747959 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f49900c1290 con 0x7f49901523d0 2024-08-18T13:45:50.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.214+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49900713a0 con 0x7f49901523d0 2024-08-18T13:45:50.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.214+0000 7f49857fa700 1 -- 172.21.15.118:0/237747959 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5367 (secure 0 0 0) 0x7f498801a020 con 0x7f49901523d0 2024-08-18T13:45:50.219 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:50.219 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:48.438475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:50.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4974060890 msgr2=0x7f4974062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:50.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4974060890 0x7f4974062d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49900701b0 tx=0x7f498c016000 comp rx=0 tx=0).stop 2024-08-18T13:45:50.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49901523d0 msgr2=0x7f49900c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49901523d0 0x7f49900c5f90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f498800a380 tx=0x7f4988007900 comp rx=0 tx=0).stop 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 shutdown_connections 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4974060890 0x7f4974062d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49901523d0 0x7f49900c5f90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f499014b940 0x7f49900c1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 --2- 172.21.15.118:0/237747959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49900be5a0 0x7f4990149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 >> 172.21.15.118:0/237747959 conn(0x7f499006c020 msgr2=0x7f499014fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 shutdown_connections 2024-08-18T13:45:50.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:50.218+0000 7f49961f9700 1 -- 172.21.15.118:0/237747959 wait complete. 2024-08-18T13:45:50.224 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 23 2024-08-18T13:45:50.239 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:45:52.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:52.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 Processor -- start 2024-08-18T13:45:52.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 -- start start 2024-08-18T13:45:52.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f956414b5a0 0x7f95640673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640c7fc0 0x7f956406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f956406d9e0 0x7f956406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95640c8730 con 0x7f956414b5a0 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95641520e0 con 0x7f956406d9e0 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640c2460 con 0x7f95640c7fc0 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f956414b5a0 0x7f95640673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f956414b5a0 0x7f95640673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58154/0 (socket says 172.21.15.118:58154) 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 -- 172.21.15.118:0/115190455 learned_addr learned my addr 172.21.15.118:0/115190455 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f9562d9d700 1 --2- 172.21.15.118:0/115190455 >> v2:172.21.15.178:3300/0 conn(0x7f956406d9e0 0x7f956406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:52.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f9561d9b700 1 --2- 172.21.15.118:0/115190455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640c7fc0 0x7f956406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 -- 172.21.15.118:0/115190455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640c7fc0 msgr2=0x7f956406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 --2- 172.21.15.118:0/115190455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640c7fc0 0x7f956406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 -- 172.21.15.118:0/115190455 >> v2:172.21.15.178:3300/0 conn(0x7f956406d9e0 msgr2=0x7f956406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 --2- 172.21.15.118:0/115190455 >> v2:172.21.15.178:3300/0 conn(0x7f956406d9e0 0x7f956406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 -- 172.21.15.118:0/115190455 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640c2720 con 0x7f956414b5a0 2024-08-18T13:45:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f9562d9d700 1 --2- 172.21.15.118:0/115190455 >> v2:172.21.15.178:3300/0 conn(0x7f956406d9e0 0x7f956406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:45:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.354+0000 7f956259c700 1 --2- 172.21.15.118:0/115190455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956414b5a0 0x7f95640673d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f956406d970 tx=0x7f9554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44851062e5da70da server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956159a700 1 -- 172.21.15.118:0/115190455 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400b640 con 0x7f956414b5a0 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956159a700 1 -- 172.21.15.118:0/115190455 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400b7a0 con 0x7f956414b5a0 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956159a700 1 -- 172.21.15.118:0/115190455 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400bb60 con 0x7f956414b5a0 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- 172.21.15.118:0/115190455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956414b5a0 msgr2=0x7f95640673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 --2- 172.21.15.118:0/115190455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956414b5a0 0x7f95640673d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f956406d970 tx=0x7f9554009250 comp rx=0 tx=0).stop 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- 172.21.15.118:0/115190455 shutdown_connections 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 --2- 172.21.15.118:0/115190455 >> v2:172.21.15.178:3300/0 conn(0x7f956406d9e0 0x7f956406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 --2- 172.21.15.118:0/115190455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95640c7fc0 0x7f956406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 --2- 172.21.15.118:0/115190455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956414b5a0 0x7f95640673d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- 172.21.15.118:0/115190455 >> 172.21.15.118:0/115190455 conn(0x7f956406c020 msgr2=0x7f95640c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- 172.21.15.118:0/115190455 shutdown_connections 2024-08-18T13:45:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- 172.21.15.118:0/115190455 wait complete. 2024-08-18T13:45:52.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 Processor -- start 2024-08-18T13:45:52.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- start start 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956406d9e0 0x7f9564141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95640c7fc0 0x7f9564165ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f956414b5a0 0x7f9564148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564150be0 con 0x7f956406d9e0 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564141810 con 0x7f95640c7fc0 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9564150990 con 0x7f956414b5a0 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956406d9e0 0x7f9564141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956406d9e0 0x7f9564141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58158/0 (socket says 172.21.15.118:58158) 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 -- 172.21.15.118:0/136622009 learned_addr learned my addr 172.21.15.118:0/136622009 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f9562d9d700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f956414b5a0 0x7f9564148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f9561d9b700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95640c7fc0 0x7f9564165ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 -- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f956414b5a0 msgr2=0x7f9564148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f956414b5a0 0x7f9564148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 -- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95640c7fc0 msgr2=0x7f9564165ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95640c7fc0 0x7f9564165ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 -- 172.21.15.118:0/136622009 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c009420 con 0x7f956406d9e0 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.358+0000 7f956259c700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956406d9e0 0x7f9564141ea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9554009f70 tx=0x7f9554017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f955a7fc700 1 -- 172.21.15.118:0/136622009 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554000d10 con 0x7f956406d9e0 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f955a7fc700 1 -- 172.21.15.118:0/136622009 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400b910 con 0x7f956406d9e0 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95540094f0 con 0x7f956406d9e0 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f955a7fc700 1 -- 172.21.15.118:0/136622009 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554026430 con 0x7f956406d9e0 2024-08-18T13:45:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9564068bb0 con 0x7f956406d9e0 2024-08-18T13:45:52.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95640713a0 con 0x7f956406d9e0 2024-08-18T13:45:52.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f955a7fc700 1 -- 172.21.15.118:0/136622009 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9554023030 con 0x7f956406d9e0 2024-08-18T13:45:52.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f955a7fc700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9544060890 0x7f9544062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:52.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f955a7fc700 1 -- 172.21.15.118:0/136622009 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f955407e860 con 0x7f956406d9e0 2024-08-18T13:45:52.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f9561d9b700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9544060890 0x7f9544062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:52.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.362+0000 7f9561d9b700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9544060890 0x7f9544062d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9564143ab0 tx=0x7f954c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:52.377 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.374+0000 7f955a7fc700 1 -- 172.21.15.118:0/136622009 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f955404e7f0 con 0x7f956406d9e0 2024-08-18T13:45:52.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.566+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f956406de30 con 0x7f956406d9e0 2024-08-18T13:45:52.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.566+0000 7f955a7fc700 1 -- 172.21.15.118:0/136622009 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5367 (secure 0 0 0) 0x7f9554051e10 con 0x7f956406d9e0 2024-08-18T13:45:52.571 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:52.571 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:48.438475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:52.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.566+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9544060890 msgr2=0x7f9544062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:52.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.566+0000 7f956359e700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9544060890 0x7f9544062d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9564143ab0 tx=0x7f954c00b040 comp rx=0 tx=0).stop 2024-08-18T13:45:52.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956406d9e0 msgr2=0x7f9564141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:52.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956406d9e0 0x7f9564141ea0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9554009f70 tx=0x7f9554017000 comp rx=0 tx=0).stop 2024-08-18T13:45:52.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 shutdown_connections 2024-08-18T13:45:52.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9544060890 0x7f9544062d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f956414b5a0 0x7f9564148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95640c7fc0 0x7f9564165ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 --2- 172.21.15.118:0/136622009 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f956406d9e0 0x7f9564141ea0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:52.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 >> 172.21.15.118:0/136622009 conn(0x7f956406c020 msgr2=0x7f9564149fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:52.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 shutdown_connections 2024-08-18T13:45:52.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:52.570+0000 7f956359e700 1 -- 172.21.15.118:0/136622009 wait complete. 2024-08-18T13:45:52.575 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 23 2024-08-18T13:45:52.585 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:45:54.586 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:54.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.698+0000 7fd77325b700 1 Processor -- start 2024-08-18T13:45:54.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.698+0000 7fd77325b700 1 -- start start 2024-08-18T13:45:54.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd76c0bd580 0x7fd76c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:54.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd76c0bea90 0x7fd76c06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:54.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c151f80 con 0x7fd76c14bcd0 2024-08-18T13:45:54.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c151990 con 0x7fd76c0bd580 2024-08-18T13:45:54.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c14c0a0 con 0x7fd76c0bea90 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd76c0bd580 0x7fd76c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd76c0bd580 0x7fd76c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36762/0 (socket says 172.21.15.118:36762) 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 -- 172.21.15.118:0/3756529503 learned_addr learned my addr 172.21.15.118:0/3756529503 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd7717f8700 1 --2- 172.21.15.118:0/3756529503 >> v2:172.21.15.178:3300/0 conn(0x7fd76c0bea90 0x7fd76c06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 -- 172.21.15.118:0/3756529503 >> v2:172.21.15.178:3300/0 conn(0x7fd76c0bea90 msgr2=0x7fd76c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 --2- 172.21.15.118:0/3756529503 >> v2:172.21.15.178:3300/0 conn(0x7fd76c0bea90 0x7fd76c06e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 -- 172.21.15.118:0/3756529503 >> v1:172.21.15.118:6789/0 conn(0x7fd76c14bcd0 legacy=0x7fd76c152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:54.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 -- 172.21.15.118:0/3756529503 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c0c2a50 con 0x7fd76c0bd580 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd7717f8700 1 --2- 172.21.15.118:0/3756529503 >> v2:172.21.15.178:3300/0 conn(0x7fd76c0bea90 0x7fd76c06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76bfff700 1 --2- 172.21.15.118:0/3756529503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c0bd580 0x7fd76c06e290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd76c06df90 tx=0x7fd75400a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12381702f7fd8435 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76b7fe700 1 -- 172.21.15.118:0/3756529503 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400cdf0 con 0x7fd76c0bd580 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76b7fe700 1 -- 172.21.15.118:0/3756529503 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75400a0d0 con 0x7fd76c0bd580 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd76b7fe700 1 -- 172.21.15.118:0/3756529503 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754008510 con 0x7fd76c0bd580 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- 172.21.15.118:0/3756529503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c0bd580 msgr2=0x7fd76c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 --2- 172.21.15.118:0/3756529503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c0bd580 0x7fd76c06e290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd76c06df90 tx=0x7fd75400a950 comp rx=0 tx=0).stop 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- 172.21.15.118:0/3756529503 shutdown_connections 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 --2- 172.21.15.118:0/3756529503 >> v2:172.21.15.178:3300/0 conn(0x7fd76c0bea90 0x7fd76c06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 --2- 172.21.15.118:0/3756529503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c0bd580 0x7fd76c06e290 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- 172.21.15.118:0/3756529503 >> 172.21.15.118:0/3756529503 conn(0x7fd76c0c7c40 msgr2=0x7fd76c0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- 172.21.15.118:0/3756529503 shutdown_connections 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.702+0000 7fd77325b700 1 -- 172.21.15.118:0/3756529503 wait complete. 2024-08-18T13:45:54.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 Processor -- start 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 -- start start 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd76c0bd580 0x7fd76c068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd76c0bea90 0x7fd76c1431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c14bcd0 0x7fd76c143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c142a50 con 0x7fd76c14bcd0 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c142d30 con 0x7fd76c0bd580 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c1619a0 con 0x7fd76c0bea90 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c14bcd0 0x7fd76c143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c14bcd0 0x7fd76c143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36770/0 (socket says 172.21.15.118:36770) 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 -- 172.21.15.118:0/772617121 learned_addr learned my addr 172.21.15.118:0/772617121 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd76bfff700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd76c0bea90 0x7fd76c1431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd770ff7700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd76c0bd580 0x7fd76c068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:54.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 -- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd76c0bea90 msgr2=0x7fd76c1431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd76c0bea90 0x7fd76c1431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 -- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd76c0bd580 msgr2=0x7fd76c068080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd76c0bd580 0x7fd76c068080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 -- 172.21.15.118:0/772617121 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75400abf0 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd770ff7700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd76c0bd580 0x7fd76c068080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd7717f8700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c14bcd0 0x7fd76c143700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e8a0 tx=0x7fd764008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd768ff9700 1 -- 172.21.15.118:0/772617121 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764013410 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd768ff9700 1 -- 172.21.15.118:0/772617121 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764013570 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c161c80 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd768ff9700 1 -- 172.21.15.118:0/772617121 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640138f0 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.706+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c162240 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.710+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76c142ee0 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.710+0000 7fd768ff9700 1 -- 172.21.15.118:0/772617121 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd764013ce0 con 0x7fd76c14bcd0 2024-08-18T13:45:54.957 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.710+0000 7fd768ff9700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd74c060860 0x7fd74c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:54.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.710+0000 7fd768ff9700 1 -- 172.21.15.118:0/772617121 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fd76407ce90 con 0x7fd76c14bcd0 2024-08-18T13:45:54.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.710+0000 7fd770ff7700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd74c060860 0x7fd74c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:54.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.710+0000 7fd770ff7700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd74c060860 0x7fd74c062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd75c001860 tx=0x7fd75c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:54.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.718+0000 7fd768ff9700 1 -- 172.21.15.118:0/772617121 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd7640508b0 con 0x7fd76c14bcd0 2024-08-18T13:45:54.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.918+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd76c0642f0 con 0x7fd76c14bcd0 2024-08-18T13:45:54.958 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.918+0000 7fd768ff9700 1 -- 172.21.15.118:0/772617121 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5367 (secure 0 0 0) 0x7fd764020090 con 0x7fd76c14bcd0 2024-08-18T13:45:54.962 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:54.962 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:54.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.918+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd74c060860 msgr2=0x7fd74c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:54.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.918+0000 7fd77325b700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd74c060860 0x7fd74c062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd75c001860 tx=0x7fd75c009040 comp rx=0 tx=0).stop 2024-08-18T13:45:54.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.918+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c14bcd0 msgr2=0x7fd76c143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:54.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.918+0000 7fd77325b700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c14bcd0 0x7fd76c143700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e8a0 tx=0x7fd764008040 comp rx=0 tx=0).stop 2024-08-18T13:45:54.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 shutdown_connections 2024-08-18T13:45:54.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd74c060860 0x7fd74c062d10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd76c14bcd0 0x7fd76c143700 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd76c0bea90 0x7fd76c1431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 --2- 172.21.15.118:0/772617121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd76c0bd580 0x7fd76c068080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:54.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 >> 172.21.15.118:0/772617121 conn(0x7fd76c0c7c40 msgr2=0x7fd76c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:54.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 shutdown_connections 2024-08-18T13:45:54.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:54.922+0000 7fd77325b700 1 -- 172.21.15.118:0/772617121 wait complete. 2024-08-18T13:45:54.965 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 24 2024-08-18T13:45:54.966 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=4805 2024-08-18T13:45:54.966 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.d 2024-08-18T13:45:58.967 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2024-08-18T13:45:58.968 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-18T13:45:58.968 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-18T13:45:58.971 INFO:tasks.ceph.mds.d:Started 2024-08-18T13:45:58.971 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:45:59.048 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-08-18T13:45:59.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.082+0000 7f5e66ad5700 1 Processor -- start 2024-08-18T13:45:59.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.082+0000 7f5e66ad5700 1 -- start start 2024-08-18T13:45:59.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5e60153690 0x7f5e600673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5e6006f2f0 0x7f5e600c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6006ee60 con 0x7f5e600c8200 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6015a640 con 0x7f5e6006f2f0 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6006f730 con 0x7f5e60153690 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5e6006f2f0 0x7f5e600c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5e6006f2f0 0x7f5e600c29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36784/0 (socket says 172.21.15.118:36784) 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 -- 172.21.15.118:0/3840671930 learned_addr learned my addr 172.21.15.118:0/3840671930 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:59.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e65ad3700 1 --2- 172.21.15.118:0/3840671930 >> v2:172.21.15.178:3300/0 conn(0x7f5e60153690 0x7f5e600673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 -- 172.21.15.118:0/3840671930 >> v2:172.21.15.178:3300/0 conn(0x7f5e60153690 msgr2=0x7f5e600673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 --2- 172.21.15.118:0/3840671930 >> v2:172.21.15.178:3300/0 conn(0x7f5e60153690 0x7f5e600673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 -- 172.21.15.118:0/3840671930 >> v1:172.21.15.118:6789/0 conn(0x7f5e600c8200 legacy=0x7f5e600c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 -- 172.21.15.118:0/3840671930 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e60153d10 con 0x7f5e6006f2f0 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e662d4700 1 --2- 172.21.15.118:0/3840671930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e6006f2f0 0x7f5e600c29b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e6006d3b0 tx=0x7f5e58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771fcc1734a2c7bb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e64ad1700 1 -- 172.21.15.118:0/3840671930 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800e070 con 0x7f5e6006f2f0 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e64ad1700 1 -- 172.21.15.118:0/3840671930 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5800ba20 con 0x7f5e6006f2f0 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e64ad1700 1 -- 172.21.15.118:0/3840671930 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580092e0 con 0x7f5e6006f2f0 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- 172.21.15.118:0/3840671930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e6006f2f0 msgr2=0x7f5e600c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/3840671930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e6006f2f0 0x7f5e600c29b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5e6006d3b0 tx=0x7f5e58009840 comp rx=0 tx=0).stop 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- 172.21.15.118:0/3840671930 shutdown_connections 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/3840671930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e6006f2f0 0x7f5e600c29b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/3840671930 >> v2:172.21.15.178:3300/0 conn(0x7f5e60153690 0x7f5e600673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- 172.21.15.118:0/3840671930 >> 172.21.15.118:0/3840671930 conn(0x7f5e6006c020 msgr2=0x7f5e600c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:59.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- 172.21.15.118:0/3840671930 shutdown_connections 2024-08-18T13:45:59.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.086+0000 7f5e66ad5700 1 -- 172.21.15.118:0/3840671930 wait complete. 2024-08-18T13:45:59.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 Processor -- start 2024-08-18T13:45:59.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 -- start start 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e6006f2f0 0x7f5e600bf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e600c8200 0x7f5e601501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e60153690 0x7f5e60150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6014fd10 con 0x7f5e60153690 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600c03a0 con 0x7f5e6006f2f0 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60158630 con 0x7f5e600c8200 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e60153690 0x7f5e60150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e60153690 0x7f5e60150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36794/0 (socket says 172.21.15.118:36794) 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 -- 172.21.15.118:0/50218791 learned_addr learned my addr 172.21.15.118:0/50218791 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e65ad3700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e6006f2f0 0x7f5e600bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:59.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e652d2700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e600c8200 0x7f5e601501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 -- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e600c8200 msgr2=0x7f5e601501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e600c8200 0x7f5e601501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 -- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e6006f2f0 msgr2=0x7f5e600bf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e6006f2f0 0x7f5e600bf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 -- 172.21.15.118:0/50218791 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e58009ae0 con 0x7f5e60153690 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e662d4700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e60153690 0x7f5e60150710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800b8c0 tx=0x7f5e58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e51ffb700 1 -- 172.21.15.118:0/50218791 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58008a30 con 0x7f5e60153690 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e51ffb700 1 -- 172.21.15.118:0/50218791 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e58008b90 con 0x7f5e60153690 2024-08-18T13:45:59.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e60158910 con 0x7f5e60153690 2024-08-18T13:45:59.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e51ffb700 1 -- 172.21.15.118:0/50218791 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58025460 con 0x7f5e60153690 2024-08-18T13:45:59.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.090+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e600be400 con 0x7f5e60153690 2024-08-18T13:45:59.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.094+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e600713a0 con 0x7f5e60153690 2024-08-18T13:45:59.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.094+0000 7f5e51ffb700 1 -- 172.21.15.118:0/50218791 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5e58003040 con 0x7f5e60153690 2024-08-18T13:45:59.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.094+0000 7f5e51ffb700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5e44060890 0x7f5e44062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:45:59.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.094+0000 7f5e51ffb700 1 -- 172.21.15.118:0/50218791 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f5e5807eb00 con 0x7f5e60153690 2024-08-18T13:45:59.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.094+0000 7f5e65ad3700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5e44060890 0x7f5e44062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:45:59.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.094+0000 7f5e65ad3700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5e44060890 0x7f5e44062d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009db0 tx=0x7f5e5c0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:45:59.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.098+0000 7f5e51ffb700 1 -- 172.21.15.118:0/50218791 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e5804ea90 con 0x7f5e60153690 2024-08-18T13:45:59.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.302+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e600c7fc0 con 0x7f5e60153690 2024-08-18T13:45:59.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.302+0000 7f5e51ffb700 1 -- 172.21.15.118:0/50218791 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5367 (secure 0 0 0) 0x7f5e58008f30 con 0x7f5e60153690 2024-08-18T13:45:59.308 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:45:59.308 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:45:59.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5e44060890 msgr2=0x7f5e44062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:59.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5e44060890 0x7f5e44062d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009db0 tx=0x7f5e5c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.310 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e60153690 msgr2=0x7f5e60150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:45:59.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e60153690 0x7f5e60150710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800b8c0 tx=0x7f5e58018000 comp rx=0 tx=0).stop 2024-08-18T13:45:59.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 shutdown_connections 2024-08-18T13:45:59.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5e44060890 0x7f5e44062d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e60153690 0x7f5e60150710 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e600c8200 0x7f5e601501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 --2- 172.21.15.118:0/50218791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e6006f2f0 0x7f5e600bf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:45:59.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 >> 172.21.15.118:0/50218791 conn(0x7f5e6006c020 msgr2=0x7f5e60159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:45:59.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 shutdown_connections 2024-08-18T13:45:59.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:45:59.306+0000 7f5e66ad5700 1 -- 172.21.15.118:0/50218791 wait complete. 2024-08-18T13:45:59.313 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 24 2024-08-18T13:45:59.322 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-08-18T13:46:01.323 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:46:01.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eba808700 1 Processor -- start 2024-08-18T13:46:01.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eba808700 1 -- start start 2024-08-18T13:46:01.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eba808700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 0x7f3eb406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eba808700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb406ac00 con 0x7f3eb406d830 2024-08-18T13:46:01.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eba808700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb406bc80 con 0x7f3eb406b5d0 2024-08-18T13:46:01.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eba808700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40bfdb0 con 0x7f3eb406dc00 2024-08-18T13:46:01.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb3fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f3eb406d830 0x7f3eb4153cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:40664/0 (socket says 172.21.15.118:40664) 2024-08-18T13:46:01.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb3fff700 1 -- 172.21.15.118:0/2518813803 learned_addr learned my addr 172.21.15.118:0/2518813803 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:46:01.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 0x7f3eb406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279454439 0 0) 0x7f3eb406ac00 con 0x7f3eb406d830 2024-08-18T13:46:01.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ea0003570 con 0x7f3eb406d830 2024-08-18T13:46:01.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 587691518 0 0) 0x7f3ea0003570 con 0x7f3eb406d830 2024-08-18T13:46:01.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 msgr2=0x7f3eb406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:46:01.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb37fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 0x7f3eb406b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 >> v1:172.21.15.178:6789/0 conn(0x7f3eb406b5d0 legacy=0x7f3eb414cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:46:01.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.434+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40bf110 con 0x7f3eb406d830 2024-08-18T13:46:01.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f3ea4001470 con 0x7f3eb406d830 2024-08-18T13:46:01.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 >> v1:172.21.15.118:6789/0 conn(0x7f3eb406d830 legacy=0x7f3eb4153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:46:01.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0003a90 0x7f3ea0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ea0004810 0x7f3ea0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.453 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 0x7f3ea00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40bfdb0 con 0x7f3ea0004810 2024-08-18T13:46:01.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea000bb40 con 0x7f3ea0003a90 2024-08-18T13:46:01.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea000bd00 con 0x7f3eb406dc00 2024-08-18T13:46:01.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ea0004810 0x7f3ea0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb3fff700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0003a90 0x7f3ea0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb8da5700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 0x7f3ea00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eab7fe700 1 -- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 msgr2=0x7f3ea00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:46:01.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 0x7f3ea00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eab7fe700 1 -- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0003a90 msgr2=0x7f3ea0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:46:01.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0003a90 0x7f3ea0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eab7fe700 1 -- 172.21.15.118:0/2518813803 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0003570 con 0x7f3ea0004810 2024-08-18T13:46:01.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb8da5700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb406dc00 0x7f3ea00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:46:01.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ea0004810 0x7f3ea0004c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4157880 tx=0x7f3e9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79585a49c9bfbc1b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:46:01.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00e710 con 0x7f3ea0004810 2024-08-18T13:46:01.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00e870 con 0x7f3ea0004810 2024-08-18T13:46:01.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb37fe700 1 -- 172.21.15.118:0/2518813803 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00ec30 con 0x7f3ea0004810 2024-08-18T13:46:01.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eba808700 1 -- 172.21.15.118:0/2518813803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ea0004810 msgr2=0x7f3ea0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:46:01.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eba808700 1 --2- 172.21.15.118:0/2518813803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ea0004810 0x7f3ea0004c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4157880 tx=0x7f3e9c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eb8da5700 1 -- 172.21.15.118:0/2518813803 reap_dead start 2024-08-18T13:46:01.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eba808700 1 -- 172.21.15.118:0/2518813803 shutdown_connections 2024-08-18T13:46:01.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eba808700 1 -- 172.21.15.118:0/2518813803 >> 172.21.15.118:0/2518813803 conn(0x7f3eb40c9d20 msgr2=0x7f3eb40ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:46:01.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eba808700 1 -- 172.21.15.118:0/2518813803 shutdown_connections 2024-08-18T13:46:01.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.438+0000 7f3eba808700 1 -- 172.21.15.118:0/2518813803 wait complete. 2024-08-18T13:46:01.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 Processor -- start 2024-08-18T13:46:01.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 -- start start 2024-08-18T13:46:01.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0004810 0x7f3eb4150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eb41529f0 0x7f3eb4151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb40c6030 0x7f3eb40c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb4150aa0 con 0x7f3eb41529f0 2024-08-18T13:46:01.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40c6a10 con 0x7f3ea0004810 2024-08-18T13:46:01.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40ca9b0 con 0x7f3eb40c6030 2024-08-18T13:46:01.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eb41529f0 0x7f3eb4151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eb41529f0 0x7f3eb4151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:36816/0 (socket says 172.21.15.118:36816) 2024-08-18T13:46:01.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 -- 172.21.15.118:0/4076559594 learned_addr learned my addr 172.21.15.118:0/4076559594 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:46:01.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eb3fff700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0004810 0x7f3eb4150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eb8da5700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb40c6030 0x7f3eb40c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 -- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb40c6030 msgr2=0x7f3eb40c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:46:01.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb40c6030 0x7f3eb40c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.461 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 -- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0004810 msgr2=0x7f3eb4150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:46:01.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0004810 0x7f3eb4150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 -- 172.21.15.118:0/4076559594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4165330 con 0x7f3eb41529f0 2024-08-18T13:46:01.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eb3fff700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0004810 0x7f3eb4150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:46:01.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eab7fe700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eb41529f0 0x7f3eb4151df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000bb10 tx=0x7f3e9c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:46:01.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eb0ff9700 1 -- 172.21.15.118:0/4076559594 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c023070 con 0x7f3eb41529f0 2024-08-18T13:46:01.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eb0ff9700 1 -- 172.21.15.118:0/4076559594 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00e9e0 con 0x7f3eb41529f0 2024-08-18T13:46:01.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb4165610 con 0x7f3eb41529f0 2024-08-18T13:46:01.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eb0ff9700 1 -- 172.21.15.118:0/4076559594 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c01e030 con 0x7f3eb41529f0 2024-08-18T13:46:01.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.442+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb4165bd0 con 0x7f3eb41529f0 2024-08-18T13:46:01.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.446+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb40648f0 con 0x7f3eb41529f0 2024-08-18T13:46:01.464 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.446+0000 7f3eb0ff9700 1 -- 172.21.15.118:0/4076559594 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3e9c013470 con 0x7f3eb41529f0 2024-08-18T13:46:01.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.446+0000 7f3eb0ff9700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e94060890 0x7f3e94062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:46:01.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.446+0000 7f3eb0ff9700 1 -- 172.21.15.118:0/4076559594 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f3e9c07e730 con 0x7f3eb41529f0 2024-08-18T13:46:01.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.446+0000 7f3eb3fff700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e94060890 0x7f3e94062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:46:01.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.446+0000 7f3eb3fff700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e94060890 0x7f3e94062d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40015a0 tx=0x7f3ea4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:46:01.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.454+0000 7f3eb0ff9700 1 -- 172.21.15.118:0/4076559594 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3e9c04e6c0 con 0x7f3eb41529f0 2024-08-18T13:46:01.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.670+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eb4150aa0 con 0x7f3eb41529f0 2024-08-18T13:46:01.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.670+0000 7f3eb0ff9700 1 -- 172.21.15.118:0/4076559594 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f3e9c051ce0 con 0x7f3eb41529f0 2024-08-18T13:46:01.675 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:46:01.675 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:46:01.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e94060890 msgr2=0x7f3e94062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e94060890 0x7f3e94062d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40015a0 tx=0x7f3ea4009040 comp rx=0 tx=0).stop 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eb41529f0 msgr2=0x7f3eb4151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eb41529f0 0x7f3eb4151df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000bb10 tx=0x7f3e9c01a000 comp rx=0 tx=0).stop 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 shutdown_connections 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3e94060890 0x7f3e94062d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eb40c6030 0x7f3eb40c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3eb41529f0 0x7f3eb4151df0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 --2- 172.21.15.118:0/4076559594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea0004810 0x7f3eb4150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 >> 172.21.15.118:0/4076559594 conn(0x7f3eb40c9d20 msgr2=0x7f3eb414b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 shutdown_connections 2024-08-18T13:46:01.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:46:01.674+0000 7f3eba808700 1 -- 172.21.15.118:0/4076559594 wait complete. 2024-08-18T13:46:01.679 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 25 2024-08-18T13:46:01.689 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2024-08-18T13:46:01.689 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 67 secs before thrashing 2024-08-18T13:46:01.733 INFO:tasks.ceph.mds.f.smithi178.stderr:2024-08-18T13:46:01.727+0000 7ff04002f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T13:46:13.185 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:46:13.187 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:46:43.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:46:43.244 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:47:08.691 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:08.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea6bca700 1 Processor -- start 2024-08-18T13:47:08.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea6bca700 1 -- start start 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea6bca700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faea0069810 0x7faea00673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea6bca700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faea0069be0 0x7faea00c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea6bca700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea00c20c0 con 0x7faea0152f40 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea6bca700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea00c2960 con 0x7faea0069810 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea6bca700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea0067910 con 0x7faea0069be0 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea5bc8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faea0069810 0x7faea00673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea5bc8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faea0069810 0x7faea00673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40070/0 (socket says 172.21.15.118:40070) 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea5bc8700 1 -- 172.21.15.118:0/683433734 learned_addr learned my addr 172.21.15.118:0/683433734 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:08.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea53c7700 1 --2- 172.21.15.118:0/683433734 >> v2:172.21.15.178:3300/0 conn(0x7faea0069be0 0x7faea00c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:08.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea5bc8700 1 -- 172.21.15.118:0/683433734 >> v2:172.21.15.178:3300/0 conn(0x7faea0069be0 msgr2=0x7faea00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:08.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea5bc8700 1 --2- 172.21.15.118:0/683433734 >> v2:172.21.15.178:3300/0 conn(0x7faea0069be0 0x7faea00c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:08.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea5bc8700 1 -- 172.21.15.118:0/683433734 >> v1:172.21.15.178:6789/0 conn(0x7faea0152f40 legacy=0x7faea014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:08.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.801+0000 7faea5bc8700 1 -- 172.21.15.118:0/683433734 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00c2df0 con 0x7faea0069810 2024-08-18T13:47:08.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea5bc8700 1 --2- 172.21.15.118:0/683433734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea0069810 0x7faea00673d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faea006ff30 tx=0x7fae9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d6db43cc52fcfd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:08.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea4bc6700 1 -- 172.21.15.118:0/683433734 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00b860 con 0x7faea0069810 2024-08-18T13:47:08.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea4bc6700 1 -- 172.21.15.118:0/683433734 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c00b9c0 con 0x7faea0069810 2024-08-18T13:47:08.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea4bc6700 1 -- 172.21.15.118:0/683433734 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00bd60 con 0x7faea0069810 2024-08-18T13:47:08.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 -- 172.21.15.118:0/683433734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea0069810 msgr2=0x7faea00673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:08.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 --2- 172.21.15.118:0/683433734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea0069810 0x7faea00673d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faea006ff30 tx=0x7fae9c009250 comp rx=0 tx=0).stop 2024-08-18T13:47:08.811 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 -- 172.21.15.118:0/683433734 shutdown_connections 2024-08-18T13:47:08.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 --2- 172.21.15.118:0/683433734 >> v2:172.21.15.178:3300/0 conn(0x7faea0069be0 0x7faea00c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:08.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 --2- 172.21.15.118:0/683433734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea0069810 0x7faea00673d0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faea006ff30 tx=0x7fae9c009250 comp rx=0 tx=0).stop 2024-08-18T13:47:08.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 -- 172.21.15.118:0/683433734 >> 172.21.15.118:0/683433734 conn(0x7faea006f4d0 msgr2=0x7faea00c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:08.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 -- 172.21.15.118:0/683433734 shutdown_connections 2024-08-18T13:47:08.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 -- 172.21.15.118:0/683433734 wait complete. 2024-08-18T13:47:08.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 Processor -- start 2024-08-18T13:47:08.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.805+0000 7faea6bca700 1 -- start start 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea0069be0 0x7faea00c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faea0152f40 0x7faea0167120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea00c75f0 0x7faea0151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0156ea0 con 0x7faea00c75f0 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0163130 con 0x7faea0069be0 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea0166c90 con 0x7faea0152f40 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea00c75f0 0x7faea0151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea00c75f0 0x7faea0151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40072/0 (socket says 172.21.15.118:40072) 2024-08-18T13:47:08.815 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea5bc8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea0069be0 0x7faea00c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 -- 172.21.15.118:0/476679984 learned_addr learned my addr 172.21.15.118:0/476679984 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea53c7700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faea0152f40 0x7faea0167120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 -- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faea0152f40 msgr2=0x7faea0167120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faea0152f40 0x7faea0167120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 -- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea0069be0 msgr2=0x7faea00c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea0069be0 0x7faea00c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 -- 172.21.15.118:0/476679984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0094f0 con 0x7faea00c75f0 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea5bc8700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea0069be0 0x7faea00c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea63c9700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea00c75f0 0x7faea0151730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c840 tx=0x7fae98001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7fae95ffb700 1 -- 172.21.15.118:0/476679984 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98012630 con 0x7faea00c75f0 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7fae95ffb700 1 -- 172.21.15.118:0/476679984 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae980127b0 con 0x7faea00c75f0 2024-08-18T13:47:08.816 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea0166680 con 0x7faea00c75f0 2024-08-18T13:47:08.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7fae95ffb700 1 -- 172.21.15.118:0/476679984 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98012b70 con 0x7faea00c75f0 2024-08-18T13:47:08.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea0163340 con 0x7faea00c75f0 2024-08-18T13:47:08.817 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.809+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea0166e40 con 0x7faea00c75f0 2024-08-18T13:47:08.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.813+0000 7fae95ffb700 1 -- 172.21.15.118:0/476679984 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fae9801a3a0 con 0x7faea00c75f0 2024-08-18T13:47:08.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.813+0000 7fae95ffb700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae84060860 0x7fae84062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:08.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.813+0000 7fae95ffb700 1 -- 172.21.15.118:0/476679984 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7fae9807d1e0 con 0x7faea00c75f0 2024-08-18T13:47:08.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.813+0000 7faea5bc8700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae84060860 0x7fae84062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:08.818 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.813+0000 7faea5bc8700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae84060860 0x7fae84062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faea006ff30 tx=0x7fae9c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:08.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:08.821+0000 7fae95ffb700 1 -- 172.21.15.118:0/476679984 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faea0166e40 con 0x7faea00c75f0 2024-08-18T13:47:09.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.037+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faea0166e00 con 0x7faea00c75f0 2024-08-18T13:47:09.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.037+0000 7fae95ffb700 1 -- 172.21.15.118:0/476679984 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6127 (secure 0 0 0) 0x7fae9804d170 con 0x7faea00c75f0 2024-08-18T13:47:09.307 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:09.308 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:09.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae84060860 msgr2=0x7fae84062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae84060860 0x7fae84062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faea006ff30 tx=0x7fae9c017000 comp rx=0 tx=0).stop 2024-08-18T13:47:09.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea00c75f0 msgr2=0x7faea0151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea00c75f0 0x7faea0151730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c840 tx=0x7fae98001670 comp rx=0 tx=0).stop 2024-08-18T13:47:09.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 shutdown_connections 2024-08-18T13:47:09.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae84060860 0x7fae84062d10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faea00c75f0 0x7faea0151730 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faea0152f40 0x7faea0167120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 --2- 172.21.15.118:0/476679984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faea0069be0 0x7faea00c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 >> 172.21.15.118:0/476679984 conn(0x7faea006f4d0 msgr2=0x7faea006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:09.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 shutdown_connections 2024-08-18T13:47:09.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.041+0000 7faea6bca700 1 -- 172.21.15.118:0/476679984 wait complete. 2024-08-18T13:47:09.313 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 25 2024-08-18T13:47:09.314 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2024-08-18T13:47:09.314 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-08-18T13:47:09.314 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:47:09.335 INFO:tasks.ceph.mds.f:Stopped 2024-08-18T13:47:09.335 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:09.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.465+0000 7f2bd6c02700 1 Processor -- start 2024-08-18T13:47:09.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.465+0000 7f2bd6c02700 1 -- start start 2024-08-18T13:47:09.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.465+0000 7f2bd6c02700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:09.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.465+0000 7f2bd6c02700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd01534f0 con 0x7f2bd00c4320 2024-08-18T13:47:09.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.465+0000 7f2bd6c02700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd0156c20 con 0x7f2bd0067d50 2024-08-18T13:47:09.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.465+0000 7f2bd6c02700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01574c0 con 0x7f2bd00c46f0 2024-08-18T13:47:09.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd5c00700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2bd00c4320 0x7f2bd005e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:33056/0 (socket says 172.21.15.118:33056) 2024-08-18T13:47:09.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd5c00700 1 -- 172.21.15.118:0/2649828294 learned_addr learned my addr 172.21.15.118:0/2649828294 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:47:09.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd53ff700 1 --2- 172.21.15.118:0/2649828294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:09.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd4bfe700 1 -- 172.21.15.118:0/2649828294 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953533774 0 0) 0x7f2bd01534f0 con 0x7f2bd00c4320 2024-08-18T13:47:09.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd4bfe700 1 -- 172.21.15.118:0/2649828294 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bbc003570 con 0x7f2bd00c4320 2024-08-18T13:47:09.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd4bfe700 1 -- 172.21.15.118:0/2649828294 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882529557 0 0) 0x7f2bd0156c20 con 0x7f2bd0067d50 2024-08-18T13:47:09.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd4bfe700 1 -- 172.21.15.118:0/2649828294 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd01534f0 con 0x7f2bd0067d50 2024-08-18T13:47:09.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd53ff700 1 -- 172.21.15.118:0/2649828294 >> v1:172.21.15.178:6789/0 conn(0x7f2bd0067d50 legacy=0x7f2bd014c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd53ff700 1 -- 172.21.15.118:0/2649828294 >> v1:172.21.15.118:6789/0 conn(0x7f2bd00c4320 legacy=0x7f2bd005e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd53ff700 1 -- 172.21.15.118:0/2649828294 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0156f00 con 0x7f2bd00c46f0 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd53ff700 1 --2- 172.21.15.118:0/2649828294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0156570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009910 tx=0x7f2bc00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c44c199fb1d06e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd4bfe700 1 -- 172.21.15.118:0/2649828294 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000b690 con 0x7f2bd00c46f0 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd4bfe700 1 -- 172.21.15.118:0/2649828294 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc000b7f0 con 0x7f2bd00c46f0 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd4bfe700 1 -- 172.21.15.118:0/2649828294 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000bb90 con 0x7f2bd00c46f0 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2649828294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 msgr2=0x7f2bd0156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2649828294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0156570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009910 tx=0x7f2bc00091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2649828294 shutdown_connections 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2649828294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0156570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2649828294 >> 172.21.15.118:0/2649828294 conn(0x7f2bd00c81e0 msgr2=0x7f2bd00c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:09.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2649828294 shutdown_connections 2024-08-18T13:47:09.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.469+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2649828294 wait complete. 2024-08-18T13:47:09.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 Processor -- start 2024-08-18T13:47:09.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 -- start start 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0067d50 0x7f2bd0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd00c4320 0x7f2bd0155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01513b0 con 0x7f2bd0067d50 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd006b420 con 0x7f2bd00c4320 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd0154930 con 0x7f2bd00c46f0 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0067d50 0x7f2bd0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0067d50 0x7f2bd0154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40078/0 (socket says 172.21.15.118:40078) 2024-08-18T13:47:09.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 -- 172.21.15.118:0/2777409351 learned_addr learned my addr 172.21.15.118:0/2777409351 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd53ff700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd00c4320 0x7f2bd0155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6401700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 -- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 msgr2=0x7f2bd0152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 -- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd00c4320 msgr2=0x7f2bd0155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd00c4320 0x7f2bd0155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 -- 172.21.15.118:0/2777409351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009420 con 0x7f2bd0067d50 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6401700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd5c00700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0067d50 0x7f2bd0154dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c7c0 tx=0x7f2bcc007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bc5ffb700 1 -- 172.21.15.118:0/2777409351 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc005b30 con 0x7f2bd0067d50 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd0166cc0 con 0x7f2bd0067d50 2024-08-18T13:47:09.482 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bc5ffb700 1 -- 172.21.15.118:0/2777409351 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc005c90 con 0x7f2bd0067d50 2024-08-18T13:47:09.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bc5ffb700 1 -- 172.21.15.118:0/2777409351 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc007300 con 0x7f2bd0067d50 2024-08-18T13:47:09.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.473+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd01542e0 con 0x7f2bd0067d50 2024-08-18T13:47:09.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.477+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0154140 con 0x7f2bd0067d50 2024-08-18T13:47:09.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.477+0000 7f2bc5ffb700 1 -- 172.21.15.118:0/2777409351 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2bcc01b3b0 con 0x7f2bd0067d50 2024-08-18T13:47:09.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.477+0000 7f2bc5ffb700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bb4060860 0x7f2bb4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:09.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.477+0000 7f2bc5ffb700 1 -- 172.21.15.118:0/2777409351 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f2bcc07c650 con 0x7f2bd0067d50 2024-08-18T13:47:09.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.477+0000 7f2bd53ff700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bb4060860 0x7f2bb4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:09.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.477+0000 7f2bd53ff700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bb4060860 0x7f2bb4062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00091e0 tx=0x7f2bc0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:09.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.485+0000 7f2bc5ffb700 1 -- 172.21.15.118:0/2777409351 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2bd0154140 con 0x7f2bd0067d50 2024-08-18T13:47:09.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.697+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd0154140 con 0x7f2bd0067d50 2024-08-18T13:47:09.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.697+0000 7f2bc5ffb700 1 -- 172.21.15.118:0/2777409351 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f2bd0154140 con 0x7f2bd0067d50 2024-08-18T13:47:09.705 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:09.705 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:09.707 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bb4060860 msgr2=0x7f2bb4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bb4060860 0x7f2bb4062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00091e0 tx=0x7f2bc0017000 comp rx=0 tx=0).stop 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0067d50 msgr2=0x7f2bd0154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0067d50 0x7f2bd0154dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c7c0 tx=0x7f2bcc007ea0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 shutdown_connections 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bb4060860 0x7f2bb4062d10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bd00c46f0 0x7f2bd0152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bd00c4320 0x7f2bd0155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 --2- 172.21.15.118:0/2777409351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0067d50 0x7f2bd0154dc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 >> 172.21.15.118:0/2777409351 conn(0x7f2bd00c81e0 msgr2=0x7f2bd014c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 shutdown_connections 2024-08-18T13:47:09.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:09.701+0000 7f2bd6c02700 1 -- 172.21.15.118:0/2777409351 wait complete. 2024-08-18T13:47:09.709 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 25 2024-08-18T13:47:09.719 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:47:11.721 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:11.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 Processor -- start 2024-08-18T13:47:11.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- start start 2024-08-18T13:47:11.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f231c06c530 0x7f231c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:11.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f231c06c900 0x7f231c14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:11.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231c0c9620 con 0x7f231c06d0f0 2024-08-18T13:47:11.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231c0c2e50 con 0x7f231c06c530 2024-08-18T13:47:11.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231c068760 con 0x7f231c06c900 2024-08-18T13:47:11.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2320d6f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f231c06c530 0x7f231c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:11.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f231bfff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f231c06d0f0 0x7f231c0c3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:33064/0 (socket says 172.21.15.118:33064) 2024-08-18T13:47:11.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f231bfff700 1 -- 172.21.15.118:0/2134982257 learned_addr learned my addr 172.21.15.118:0/2134982257 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321570700 1 --2- 172.21.15.118:0/2134982257 >> v2:172.21.15.178:3300/0 conn(0x7f231c06c900 0x7f231c14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f231b7fe700 1 -- 172.21.15.118:0/2134982257 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317968965 0 0) 0x7f231c0c9620 con 0x7f231c06d0f0 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f231b7fe700 1 -- 172.21.15.118:0/2134982257 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2304003570 con 0x7f231c06d0f0 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2320d6f700 1 -- 172.21.15.118:0/2134982257 >> v2:172.21.15.178:3300/0 conn(0x7f231c06c900 msgr2=0x7f231c14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2320d6f700 1 --2- 172.21.15.118:0/2134982257 >> v2:172.21.15.178:3300/0 conn(0x7f231c06c900 0x7f231c14eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2320d6f700 1 -- 172.21.15.118:0/2134982257 >> v1:172.21.15.118:6789/0 conn(0x7f231c06d0f0 legacy=0x7f231c0c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2320d6f700 1 -- 172.21.15.118:0/2134982257 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c067b50 con 0x7f231c06c530 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2320d6f700 1 --2- 172.21.15.118:0/2134982257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c530 0x7f231c155d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5370 tx=0x7f230c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d124ac8938e7619e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f231b7fe700 1 -- 172.21.15.118:0/2134982257 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c011630 con 0x7f231c06c530 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f231b7fe700 1 -- 172.21.15.118:0/2134982257 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c0117b0 con 0x7f231c06c530 2024-08-18T13:47:11.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f231b7fe700 1 -- 172.21.15.118:0/2134982257 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c011b70 con 0x7f231c06c530 2024-08-18T13:47:11.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- 172.21.15.118:0/2134982257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c530 msgr2=0x7f231c155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:11.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 --2- 172.21.15.118:0/2134982257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c530 0x7f231c155d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5370 tx=0x7f230c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:11.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- 172.21.15.118:0/2134982257 shutdown_connections 2024-08-18T13:47:11.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 --2- 172.21.15.118:0/2134982257 >> v2:172.21.15.178:3300/0 conn(0x7f231c06c900 0x7f231c14eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:11.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 --2- 172.21.15.118:0/2134982257 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c530 0x7f231c155d80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:11.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- 172.21.15.118:0/2134982257 >> 172.21.15.118:0/2134982257 conn(0x7f231c06f420 msgr2=0x7f231c0c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:11.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- 172.21.15.118:0/2134982257 shutdown_connections 2024-08-18T13:47:11.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.837+0000 7f2321d71700 1 -- 172.21.15.118:0/2134982257 wait complete. 2024-08-18T13:47:11.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 Processor -- start 2024-08-18T13:47:11.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 -- start start 2024-08-18T13:47:11.848 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f231c06c530 0x7f231c0c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:11.848 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c900 0x7f231c157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:11.848 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f231c06d0f0 0x7f231c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:11.848 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c154300 con 0x7f231c06c900 2024-08-18T13:47:11.848 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1683f0 con 0x7f231c06c530 2024-08-18T13:47:11.848 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321d71700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c168f70 con 0x7f231c06d0f0 2024-08-18T13:47:11.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c900 0x7f231c157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:11.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c900 0x7f231c157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40104/0 (socket says 172.21.15.118:40104) 2024-08-18T13:47:11.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 -- 172.21.15.118:0/2566003630 learned_addr learned my addr 172.21.15.118:0/2566003630 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:11.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2320d6f700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f231c06c530 0x7f231c0c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:11.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f2321570700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f231c06d0f0 0x7f231c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:11.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 -- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f231c06d0f0 msgr2=0x7f231c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:11.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f231c06d0f0 0x7f231c158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:11.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 -- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f231c06c530 msgr2=0x7f231c0c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:11.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f231c06c530 0x7f231c0c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:11.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.841+0000 7f231bfff700 1 -- 172.21.15.118:0/2566003630 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c009420 con 0x7f231c06c900 2024-08-18T13:47:11.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f231bfff700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c900 0x7f231c157b30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f231000c590 tx=0x7f2310007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:11.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2318ff9700 1 -- 172.21.15.118:0/2566003630 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310005c00 con 0x7f231c06c900 2024-08-18T13:47:11.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2318ff9700 1 -- 172.21.15.118:0/2566003630 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2310005d60 con 0x7f231c06c900 2024-08-18T13:47:11.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c159390 con 0x7f231c06c900 2024-08-18T13:47:11.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2318ff9700 1 -- 172.21.15.118:0/2566003630 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310007300 con 0x7f231c06c900 2024-08-18T13:47:11.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c168920 con 0x7f231c06c900 2024-08-18T13:47:11.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231c0713a0 con 0x7f231c06c900 2024-08-18T13:47:11.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2318ff9700 1 -- 172.21.15.118:0/2566003630 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f231001b3b0 con 0x7f231c06c900 2024-08-18T13:47:11.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2318ff9700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2308060890 0x7f2308062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:11.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2320d6f700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2308060890 0x7f2308062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:11.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2318ff9700 1 -- 172.21.15.118:0/2566003630 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f231007c510 con 0x7f231c06c900 2024-08-18T13:47:11.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.845+0000 7f2320d6f700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2308060890 0x7f2308062d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5370 tx=0x7f230c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:11.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:11.853+0000 7f2318ff9700 1 -- 172.21.15.118:0/2566003630 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f231004c4a0 con 0x7f231c06c900 2024-08-18T13:47:12.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.069+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f231c0bfae0 con 0x7f231c06c900 2024-08-18T13:47:12.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.069+0000 7f2318ff9700 1 -- 172.21.15.118:0/2566003630 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f231004fac0 con 0x7f231c06c900 2024-08-18T13:47:12.077 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:12.077 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:12.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2308060890 msgr2=0x7f2308062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:12.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2308060890 0x7f2308062d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5370 tx=0x7f230c017000 comp rx=0 tx=0).stop 2024-08-18T13:47:12.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c900 msgr2=0x7f231c157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:12.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c900 0x7f231c157b30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f231000c590 tx=0x7f2310007dd0 comp rx=0 tx=0).stop 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 shutdown_connections 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2308060890 0x7f2308062d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f231c06d0f0 0x7f231c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f231c06c900 0x7f231c157b30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 --2- 172.21.15.118:0/2566003630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f231c06c530 0x7f231c0c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 >> 172.21.15.118:0/2566003630 conn(0x7f231c06f420 msgr2=0x7f231c15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 shutdown_connections 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:12.073+0000 7f2321d71700 1 -- 172.21.15.118:0/2566003630 wait complete. 2024-08-18T13:47:12.080 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 25 2024-08-18T13:47:12.091 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:47:13.286 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:47:13.288 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:47:14.092 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:14.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.201+0000 7f5213fff700 1 Processor -- start 2024-08-18T13:47:14.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.201+0000 7f5213fff700 1 -- start start 2024-08-18T13:47:14.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f521414b840 0x7f5214155580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:14.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5214069fc0 0x7f521406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:14.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521406b180 0x7f521406c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:14.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52140c1d80 con 0x7f5214069fc0 2024-08-18T13:47:14.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52140c2de0 con 0x7f521414b840 2024-08-18T13:47:14.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c30c0 con 0x7f521406b180 2024-08-18T13:47:14.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5214069fc0 0x7f521406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5214069fc0 0x7f521406bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:32826/0 (socket says 172.21.15.118:32826) 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 -- 172.21.15.118:0/4137003242 learned_addr learned my addr 172.21.15.118:0/4137003242 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52137fe700 1 --2- 172.21.15.118:0/4137003242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521406b180 0x7f521406c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5212ffd700 1 --2- 172.21.15.118:0/4137003242 >> v2:172.21.15.178:3300/0 conn(0x7f521414b840 0x7f5214155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 -- 172.21.15.118:0/4137003242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521406b180 msgr2=0x7f521406c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 --2- 172.21.15.118:0/4137003242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521406b180 0x7f521406c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 -- 172.21.15.118:0/4137003242 >> v2:172.21.15.178:3300/0 conn(0x7f521414b840 msgr2=0x7f5214155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 --2- 172.21.15.118:0/4137003242 >> v2:172.21.15.178:3300/0 conn(0x7f521414b840 0x7f5214155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 -- 172.21.15.118:0/4137003242 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521406a490 con 0x7f5214069fc0 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f52127fc700 1 --2- 172.21.15.118:0/4137003242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521406bca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5214069580 tx=0x7f5204009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3305b259a9ea1201 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5211ffb700 1 -- 172.21.15.118:0/4137003242 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520400b860 con 0x7f5214069fc0 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5211ffb700 1 -- 172.21.15.118:0/4137003242 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520400b9c0 con 0x7f5214069fc0 2024-08-18T13:47:14.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5211ffb700 1 -- 172.21.15.118:0/4137003242 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520400bd60 con 0x7f5214069fc0 2024-08-18T13:47:14.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- 172.21.15.118:0/4137003242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 msgr2=0x7f521406bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:14.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 --2- 172.21.15.118:0/4137003242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521406bca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5214069580 tx=0x7f5204009250 comp rx=0 tx=0).stop 2024-08-18T13:47:14.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- 172.21.15.118:0/4137003242 shutdown_connections 2024-08-18T13:47:14.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 --2- 172.21.15.118:0/4137003242 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521406b180 0x7f521406c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 --2- 172.21.15.118:0/4137003242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521406bca0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 --2- 172.21.15.118:0/4137003242 >> v2:172.21.15.178:3300/0 conn(0x7f521414b840 0x7f5214155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- 172.21.15.118:0/4137003242 >> 172.21.15.118:0/4137003242 conn(0x7f52140c81e0 msgr2=0x7f52140c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:14.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- 172.21.15.118:0/4137003242 shutdown_connections 2024-08-18T13:47:14.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.205+0000 7f5213fff700 1 -- 172.21.15.118:0/4137003242 wait complete. 2024-08-18T13:47:14.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 Processor -- start 2024-08-18T13:47:14.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 -- start start 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521406b180 0x7f52141415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521414b840 0x7f5214141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521414f820 con 0x7f5214069fc0 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521414c2e0 con 0x7f521406b180 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521414eea0 con 0x7f521414b840 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521414d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:32830/0 (socket says 172.21.15.118:32830) 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 -- 172.21.15.118:0/556686153 learned_addr learned my addr 172.21.15.118:0/556686153 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f52127fc700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521406b180 0x7f52141415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f52137fe700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521414b840 0x7f5214141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:14.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 -- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521414b840 msgr2=0x7f5214141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521414b840 0x7f5214141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 -- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521406b180 msgr2=0x7f52141415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521406b180 0x7f52141415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 -- 172.21.15.118:0/556686153 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040094f0 con 0x7f5214069fc0 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5212ffd700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521414d630 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f520c009fd0 tx=0x7f520c009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f51faffd700 1 -- 172.21.15.118:0/556686153 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c00ebf0 con 0x7f5214069fc0 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f51faffd700 1 -- 172.21.15.118:0/556686153 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c00ed50 con 0x7f5214069fc0 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521414db70 con 0x7f5214069fc0 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f51faffd700 1 -- 172.21.15.118:0/556686153 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c014670 con 0x7f5214069fc0 2024-08-18T13:47:14.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.209+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521414c000 con 0x7f5214069fc0 2024-08-18T13:47:14.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.213+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521414f630 con 0x7f5214069fc0 2024-08-18T13:47:14.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.213+0000 7f51faffd700 1 -- 172.21.15.118:0/556686153 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f520c014a20 con 0x7f5214069fc0 2024-08-18T13:47:14.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.213+0000 7f51faffd700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5200060890 0x7f5200062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:14.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.213+0000 7f51faffd700 1 -- 172.21.15.118:0/556686153 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f520c07d590 con 0x7f5214069fc0 2024-08-18T13:47:14.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.213+0000 7f52127fc700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5200060890 0x7f5200062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:14.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.217+0000 7f52127fc700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5200060890 0x7f5200062d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f521406a3c0 tx=0x7f5204017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:14.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.221+0000 7f51faffd700 1 -- 172.21.15.118:0/556686153 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f520c04d520 con 0x7f5214069fc0 2024-08-18T13:47:14.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.477+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f521414d260 con 0x7f5214069fc0 2024-08-18T13:47:14.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f51faffd700 1 -- 172.21.15.118:0/556686153 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f520c050b40 con 0x7f5214069fc0 2024-08-18T13:47:14.481 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:14.481 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:14.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5200060890 msgr2=0x7f5200062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5200060890 0x7f5200062d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f521406a3c0 tx=0x7f5204017000 comp rx=0 tx=0).stop 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 msgr2=0x7f521414d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521414d630 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f520c009fd0 tx=0x7f520c009880 comp rx=0 tx=0).stop 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 shutdown_connections 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5200060890 0x7f5200062d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f521414b840 0x7f5214141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521406b180 0x7f52141415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 --2- 172.21.15.118:0/556686153 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5214069fc0 0x7f521414d630 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 >> 172.21.15.118:0/556686153 conn(0x7f52140c81e0 msgr2=0x7f5214067580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 shutdown_connections 2024-08-18T13:47:14.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:14.481+0000 7f5213fff700 1 -- 172.21.15.118:0/556686153 wait complete. 2024-08-18T13:47:14.486 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 25 2024-08-18T13:47:14.501 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:47:16.501 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:16.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.612+0000 7facd1089700 1 Processor -- start 2024-08-18T13:47:16.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.612+0000 7facd1089700 1 -- start start 2024-08-18T13:47:16.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.612+0000 7facd1089700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faccc0c83e0 0x7faccc152500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:16.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.612+0000 7facd1089700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc0681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:16.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0c9150 con 0x7faccc0673d0 2024-08-18T13:47:16.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc067d70 con 0x7faccc0c83e0 2024-08-18T13:47:16.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc152a40 con 0x7faccc0c87b0 2024-08-18T13:47:16.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccbfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faccc0c83e0 0x7faccc152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:16.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccb7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc0681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:16.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccbfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faccc0c83e0 0x7faccc152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:47336/0 (socket says 172.21.15.118:47336) 2024-08-18T13:47:16.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccbfff700 1 -- 172.21.15.118:0/3712832648 learned_addr learned my addr 172.21.15.118:0/3712832648 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:16.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd0888700 1 --1- 172.21.15.118:0/3712832648 >> v1:172.21.15.178:6789/0 conn(0x7faccc0673d0 0x7faccc155050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:56598/0 (socket says 172.21.15.118:56598) 2024-08-18T13:47:16.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccb7fe700 1 -- 172.21.15.118:0/3712832648 >> v2:172.21.15.178:3300/0 conn(0x7faccc0c83e0 msgr2=0x7faccc152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:16.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccb7fe700 1 --2- 172.21.15.118:0/3712832648 >> v2:172.21.15.178:3300/0 conn(0x7faccc0c83e0 0x7faccc152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccb7fe700 1 -- 172.21.15.118:0/3712832648 >> v1:172.21.15.178:6789/0 conn(0x7faccc0673d0 legacy=0x7faccc155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:16.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccb7fe700 1 -- 172.21.15.118:0/3712832648 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc06e5d0 con 0x7faccc0c87b0 2024-08-18T13:47:16.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccbfff700 1 --2- 172.21.15.118:0/3712832648 >> v2:172.21.15.178:3300/0 conn(0x7faccc0c83e0 0x7faccc152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:47:16.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccb7fe700 1 --2- 172.21.15.118:0/3712832648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc0681c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facc00098e0 tx=0x7facc00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=684de3e90225cf96 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:16.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccaffd700 1 -- 172.21.15.118:0/3712832648 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000b640 con 0x7faccc0c87b0 2024-08-18T13:47:16.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccaffd700 1 -- 172.21.15.118:0/3712832648 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc000b7a0 con 0x7faccc0c87b0 2024-08-18T13:47:16.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7faccaffd700 1 -- 172.21.15.118:0/3712832648 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc000bb40 con 0x7faccc0c87b0 2024-08-18T13:47:16.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- 172.21.15.118:0/3712832648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 msgr2=0x7faccc0681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:16.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 --2- 172.21.15.118:0/3712832648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc0681c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facc00098e0 tx=0x7facc00091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- 172.21.15.118:0/3712832648 shutdown_connections 2024-08-18T13:47:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 --2- 172.21.15.118:0/3712832648 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc0681c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 --2- 172.21.15.118:0/3712832648 >> v2:172.21.15.178:3300/0 conn(0x7faccc0c83e0 0x7faccc152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- 172.21.15.118:0/3712832648 >> 172.21.15.118:0/3712832648 conn(0x7faccc06bdf0 msgr2=0x7faccc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- 172.21.15.118:0/3712832648 shutdown_connections 2024-08-18T13:47:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.616+0000 7facd1089700 1 -- 172.21.15.118:0/3712832648 wait complete. 2024-08-18T13:47:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 Processor -- start 2024-08-18T13:47:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 -- start start 2024-08-18T13:47:16.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0673d0 0x7faccc13ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:16.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c83e0 0x7faccc14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:16.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:16.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc162200 con 0x7faccc0673d0 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc151830 con 0x7faccc0c83e0 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc161d70 con 0x7faccc0c87b0 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0673d0 0x7faccc13ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0673d0 0x7faccc13ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:32836/0 (socket says 172.21.15.118:32836) 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 -- 172.21.15.118:0/1582840938 learned_addr learned my addr 172.21.15.118:0/1582840938 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd0888700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccb7fe700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c83e0 0x7faccc14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 -- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 msgr2=0x7faccc15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc15aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 -- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c83e0 msgr2=0x7faccc14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:16.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c83e0 0x7faccc14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 -- 172.21.15.118:0/1582840938 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc00b780 con 0x7faccc0673d0 2024-08-18T13:47:16.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7faccbfff700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0673d0 0x7faccc13ff40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7facbc0100f0 tx=0x7facbc00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:16.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facb3fff700 1 -- 172.21.15.118:0/1582840938 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc012040 con 0x7faccc0673d0 2024-08-18T13:47:16.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facb3fff700 1 -- 172.21.15.118:0/1582840938 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc010c30 con 0x7faccc0673d0 2024-08-18T13:47:16.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc0009420 con 0x7faccc0673d0 2024-08-18T13:47:16.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facb3fff700 1 -- 172.21.15.118:0/1582840938 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc018410 con 0x7faccc0673d0 2024-08-18T13:47:16.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.620+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc161440 con 0x7faccc0673d0 2024-08-18T13:47:16.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.624+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc0713a0 con 0x7faccc0673d0 2024-08-18T13:47:16.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.624+0000 7facb3fff700 1 -- 172.21.15.118:0/1582840938 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7facbc018800 con 0x7faccc0673d0 2024-08-18T13:47:16.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.624+0000 7facb3fff700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb8060860 0x7facb8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:16.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.624+0000 7facb3fff700 1 -- 172.21.15.118:0/1582840938 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7facbc07f4a0 con 0x7faccc0673d0 2024-08-18T13:47:16.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.624+0000 7faccb7fe700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb8060860 0x7facb8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:16.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.624+0000 7faccb7fe700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb8060860 0x7facb8062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7facc00091e0 tx=0x7facc0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:16.634 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.632+0000 7facb3fff700 1 -- 172.21.15.118:0/1582840938 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7facbc04f430 con 0x7faccc0673d0 2024-08-18T13:47:16.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.856+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faccc14aee0 con 0x7faccc0673d0 2024-08-18T13:47:16.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.856+0000 7facb3fff700 1 -- 172.21.15.118:0/1582840938 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6127 (secure 0 0 0) 0x7facbc052a50 con 0x7faccc0673d0 2024-08-18T13:47:16.858 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:16.858 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.856+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb8060860 msgr2=0x7facb8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.856+0000 7facd1089700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb8060860 0x7facb8062d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7facc00091e0 tx=0x7facc0017000 comp rx=0 tx=0).stop 2024-08-18T13:47:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.856+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0673d0 msgr2=0x7faccc13ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.856+0000 7facd1089700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0673d0 0x7faccc13ff40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7facbc0100f0 tx=0x7facbc00f040 comp rx=0 tx=0).stop 2024-08-18T13:47:16.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 shutdown_connections 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7facb8060860 0x7facb8062d10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0c87b0 0x7faccc15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0c83e0 0x7faccc14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 --2- 172.21.15.118:0/1582840938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faccc0673d0 0x7faccc13ff40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 >> 172.21.15.118:0/1582840938 conn(0x7faccc06bdf0 msgr2=0x7faccc14b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 shutdown_connections 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:16.860+0000 7facd1089700 1 -- 172.21.15.118:0/1582840938 wait complete. 2024-08-18T13:47:16.862 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 25 2024-08-18T13:47:16.874 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:47:18.876 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:18.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b30e5700 1 Processor -- start 2024-08-18T13:47:18.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b30e5700 1 -- start start 2024-08-18T13:47:18.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b30e5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f74ac05e6b0 0x7f74ac152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:18.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b30e5700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac067470 con 0x7f74ac0c0c50 2024-08-18T13:47:18.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b30e5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac067710 con 0x7f74ac14bf60 2024-08-18T13:47:18.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b30e5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac068370 con 0x7f74ac05e6b0 2024-08-18T13:47:18.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b28e4700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f74ac0c0c50 0x7f74ac0c10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:56902/0 (socket says 172.21.15.118:56902) 2024-08-18T13:47:18.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b28e4700 1 -- 172.21.15.118:0/1028840153 learned_addr learned my addr 172.21.15.118:0/1028840153 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:47:18.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b18e2700 1 --2- 172.21.15.118:0/1028840153 >> v2:172.21.15.178:3300/0 conn(0x7f74ac05e6b0 0x7f74ac152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:18.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b10e1700 1 -- 172.21.15.118:0/1028840153 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046190769 0 0) 0x7f74ac067470 con 0x7f74ac0c0c50 2024-08-18T13:47:18.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b10e1700 1 -- 172.21.15.118:0/1028840153 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7498003570 con 0x7f74ac0c0c50 2024-08-18T13:47:18.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b18e2700 1 -- 172.21.15.118:0/1028840153 >> v1:172.21.15.178:6789/0 conn(0x7f74ac14bf60 legacy=0x7f74ac06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:18.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b18e2700 1 -- 172.21.15.118:0/1028840153 >> v1:172.21.15.118:6789/0 conn(0x7f74ac0c0c50 legacy=0x7f74ac0c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:18.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.992+0000 7f74b18e2700 1 -- 172.21.15.118:0/1028840153 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac06cb10 con 0x7f74ac05e6b0 2024-08-18T13:47:18.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b18e2700 1 --2- 172.21.15.118:0/1028840153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac05e6b0 0x7f74ac152840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098e0 tx=0x7f749c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c876a5e02930f0dd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:18.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b10e1700 1 -- 172.21.15.118:0/1028840153 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00b640 con 0x7f74ac05e6b0 2024-08-18T13:47:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b10e1700 1 -- 172.21.15.118:0/1028840153 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c00b7a0 con 0x7f74ac05e6b0 2024-08-18T13:47:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b10e1700 1 -- 172.21.15.118:0/1028840153 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00bb20 con 0x7f74ac05e6b0 2024-08-18T13:47:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- 172.21.15.118:0/1028840153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac05e6b0 msgr2=0x7f74ac152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:18.998 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 --2- 172.21.15.118:0/1028840153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac05e6b0 0x7f74ac152840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098e0 tx=0x7f749c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- 172.21.15.118:0/1028840153 shutdown_connections 2024-08-18T13:47:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 --2- 172.21.15.118:0/1028840153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac05e6b0 0x7f74ac152840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- 172.21.15.118:0/1028840153 >> 172.21.15.118:0/1028840153 conn(0x7f74ac0c81e0 msgr2=0x7f74ac0c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- 172.21.15.118:0/1028840153 shutdown_connections 2024-08-18T13:47:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- 172.21.15.118:0/1028840153 wait complete. 2024-08-18T13:47:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 Processor -- start 2024-08-18T13:47:18.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- start start 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ac05e6b0 0x7f74ac0c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac0c0c50 0x7f74ac141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac14bf60 0x7f74ac142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac152d80 con 0x7f74ac14bf60 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac151c30 con 0x7f74ac0c0c50 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b30e5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac14b020 con 0x7f74ac05e6b0 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b28e4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac14bf60 0x7f74ac142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b28e4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac14bf60 0x7f74ac142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:32842/0 (socket says 172.21.15.118:32842) 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:18.996+0000 7f74b28e4700 1 -- 172.21.15.118:0/3685678449 learned_addr learned my addr 172.21.15.118:0/3685678449 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b18e2700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac0c0c50 0x7f74ac141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b20e3700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ac05e6b0 0x7f74ac0c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:19.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b28e4700 1 -- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ac05e6b0 msgr2=0x7f74ac0c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:19.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b28e4700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ac05e6b0 0x7f74ac0c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b28e4700 1 -- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac0c0c50 msgr2=0x7f74ac141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b28e4700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac0c0c50 0x7f74ac141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b28e4700 1 -- 172.21.15.118:0/3685678449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c009420 con 0x7f74ac14bf60 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b28e4700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac14bf60 0x7f74ac142470 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f74a4001460 tx=0x7f74a4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74a27fc700 1 -- 172.21.15.118:0/3685678449 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400ce80 con 0x7f74ac14bf60 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74a27fc700 1 -- 172.21.15.118:0/3685678449 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a400ab70 con 0x7f74ac14bf60 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac14b270 con 0x7f74ac14bf60 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74a27fc700 1 -- 172.21.15.118:0/3685678449 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40070f0 con 0x7f74ac14bf60 2024-08-18T13:47:19.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac1515d0 con 0x7f74ac14bf60 2024-08-18T13:47:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac0713a0 con 0x7f74ac14bf60 2024-08-18T13:47:19.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74a27fc700 1 -- 172.21.15.118:0/3685678449 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f74a4007520 con 0x7f74ac14bf60 2024-08-18T13:47:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74a27fc700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7490060960 0x7f7490062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74a27fc700 1 -- 172.21.15.118:0/3685678449 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6917+0+0 (secure 0 0 0) 0x7f74a407d400 con 0x7f74ac14bf60 2024-08-18T13:47:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.000+0000 7f74b20e3700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7490060960 0x7f7490062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:19.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.004+0000 7f74b20e3700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7490060960 0x7f7490062e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f74a80037c0 tx=0x7f74a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:19.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.008+0000 7f74a27fc700 1 -- 172.21.15.118:0/3685678449 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f74a404d4f0 con 0x7f74ac14bf60 2024-08-18T13:47:19.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.232+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74ac0c10d0 con 0x7f74ac14bf60 2024-08-18T13:47:19.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.232+0000 7f74a27fc700 1 -- 172.21.15.118:0/3685678449 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6127 (secure 0 0 0) 0x7f74a4050b10 con 0x7f74ac14bf60 2024-08-18T13:47:19.236 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:19.236 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:45:54.468932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5209},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5209":{"gid":5209,"name":"f","rank":0,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.178:6835/679522662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":679522662},{"type":"v1","addr":"172.21.15.178:6835","nonce":679522662}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:19.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7490060960 msgr2=0x7f7490062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7490060960 0x7f7490062e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f74a80037c0 tx=0x7f74a8009040 comp rx=0 tx=0).stop 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac14bf60 msgr2=0x7f74ac142470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac14bf60 0x7f74ac142470 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f74a4001460 tx=0x7f74a4001ae0 comp rx=0 tx=0).stop 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 shutdown_connections 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7490060960 0x7f7490062e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac14bf60 0x7f74ac142470 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ac0c0c50 0x7f74ac141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 --2- 172.21.15.118:0/3685678449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ac05e6b0 0x7f74ac0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 >> 172.21.15.118:0/3685678449 conn(0x7f74ac0c81e0 msgr2=0x7f74ac06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 shutdown_connections 2024-08-18T13:47:19.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:19.236+0000 7f74b30e5700 1 -- 172.21.15.118:0/3685678449 wait complete. 2024-08-18T13:47:19.240 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 25 2024-08-18T13:47:19.256 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:47:21.257 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:21.377 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 Processor -- start 2024-08-18T13:47:21.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- start start 2024-08-18T13:47:21.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcdec0aebc0 0x7fcdec0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcdec0aef90 0x7fcdec0abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0c6880 0x7fcdec0c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec155140 con 0x7fcdec0aef90 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec0af660 con 0x7fcdec0aebc0 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec155330 con 0x7fcdec0c6880 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcdec0aef90 0x7fcdec0abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcdec0aef90 0x7fcdec0abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:32852/0 (socket says 172.21.15.118:32852) 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 -- 172.21.15.118:0/2594540065 learned_addr learned my addr 172.21.15.118:0/2594540065 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdeb59e700 1 --2- 172.21.15.118:0/2594540065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0c6880 0x7fcdec0c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdead9d700 1 --2- 172.21.15.118:0/2594540065 >> v2:172.21.15.178:3300/0 conn(0x7fcdec0aebc0 0x7fcdec0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 -- 172.21.15.118:0/2594540065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0c6880 msgr2=0x7fcdec0c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 --2- 172.21.15.118:0/2594540065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0c6880 0x7fcdec0c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 -- 172.21.15.118:0/2594540065 >> v2:172.21.15.178:3300/0 conn(0x7fcdec0aebc0 msgr2=0x7fcdec0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:21.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 --2- 172.21.15.118:0/2594540065 >> v2:172.21.15.178:3300/0 conn(0x7fcdec0aebc0 0x7fcdec0b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 -- 172.21.15.118:0/2594540065 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec1555d0 con 0x7fcdec0aef90 2024-08-18T13:47:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdea59c700 1 --2- 172.21.15.118:0/2594540065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec0abad0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0c8570 tx=0x7fcde0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e9a0308282d04d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcde9d9b700 1 -- 172.21.15.118:0/2594540065 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000b860 con 0x7fcdec0aef90 2024-08-18T13:47:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcde9d9b700 1 -- 172.21.15.118:0/2594540065 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde000b9c0 con 0x7fcdec0aef90 2024-08-18T13:47:21.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcde9d9b700 1 -- 172.21.15.118:0/2594540065 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000bd60 con 0x7fcdec0aef90 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- 172.21.15.118:0/2594540065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 msgr2=0x7fcdec0abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 --2- 172.21.15.118:0/2594540065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec0abad0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0c8570 tx=0x7fcde0009250 comp rx=0 tx=0).stop 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- 172.21.15.118:0/2594540065 shutdown_connections 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 --2- 172.21.15.118:0/2594540065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0c6880 0x7fcdec0c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 --2- 172.21.15.118:0/2594540065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec0abad0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 --2- 172.21.15.118:0/2594540065 >> v2:172.21.15.178:3300/0 conn(0x7fcdec0aebc0 0x7fcdec0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- 172.21.15.118:0/2594540065 >> 172.21.15.118:0/2594540065 conn(0x7fcdec0cbba0 msgr2=0x7fcdec0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- 172.21.15.118:0/2594540065 shutdown_connections 2024-08-18T13:47:21.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.376+0000 7fcdf087b700 1 -- 172.21.15.118:0/2594540065 wait complete. 2024-08-18T13:47:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 Processor -- start 2024-08-18T13:47:21.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 -- start start 2024-08-18T13:47:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0aebc0 0x7fcdec14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.384 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec0c6880 0x7fcdec0682b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec069d50 con 0x7fcdec0aef90 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0c3630 con 0x7fcdec0c6880 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec165dc0 con 0x7fcdec0aebc0 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec067d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:32860/0 (socket says 172.21.15.118:32860) 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 -- 172.21.15.118:0/3066966613 learned_addr learned my addr 172.21.15.118:0/3066966613 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdead9d700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0aebc0 0x7fcdec14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdeb59e700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec0c6880 0x7fcdec0682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 -- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0aebc0 msgr2=0x7fcdec14fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0aebc0 0x7fcdec14fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 -- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec0c6880 msgr2=0x7fcdec0682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec0c6880 0x7fcdec0682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 -- 172.21.15.118:0/3066966613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00094f0 con 0x7fcdec0aef90 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdea59c700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec067d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcdec089920 tx=0x7fcde0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdd2ffd700 1 -- 172.21.15.118:0/3066966613 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0024440 con 0x7fcdec0aef90 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdd2ffd700 1 -- 172.21.15.118:0/3066966613 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde00245a0 con 0x7fcdec0aef90 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec0678e0 con 0x7fcdec0aef90 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdd2ffd700 1 -- 172.21.15.118:0/3066966613 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0024960 con 0x7fcdec0aef90 2024-08-18T13:47:21.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.380+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec069a70 con 0x7fcdec0aef90 2024-08-18T13:47:21.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.384+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec067a70 con 0x7fcdec0aef90 2024-08-18T13:47:21.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.384+0000 7fcdd2ffd700 1 -- 172.21.15.118:0/3066966613 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcde001a420 con 0x7fcdec0aef90 2024-08-18T13:47:21.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.384+0000 7fcdd2ffd700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdd8060860 0x7fcdd8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.384+0000 7fcdd2ffd700 1 -- 172.21.15.118:0/3066966613 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7fcde007ee60 con 0x7fcdec0aef90 2024-08-18T13:47:21.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.384+0000 7fcdead9d700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdd8060860 0x7fcdd8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.384+0000 7fcdead9d700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdd8060860 0x7fcdd8062d10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006fd0 tx=0x7fcddc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:21.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.392+0000 7fcdd2ffd700 1 -- 172.21.15.118:0/3066966613 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcde004eda0 con 0x7fcdec0aef90 2024-08-18T13:47:21.610 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.608+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdec0713a0 con 0x7fcdec0aef90 2024-08-18T13:47:21.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.608+0000 7fcdd2ffd700 1 -- 172.21.15.118:0/3066966613 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5369 (secure 0 0 0) 0x7fcde00523c0 con 0x7fcdec0aef90 2024-08-18T13:47:21.611 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:21.611 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:20.593666+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:21.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdd8060860 msgr2=0x7fcdd8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdd8060860 0x7fcdd8062d10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006fd0 tx=0x7fcddc008040 comp rx=0 tx=0).stop 2024-08-18T13:47:21.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 msgr2=0x7fcdec067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec067d70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcdec089920 tx=0x7fcde0017000 comp rx=0 tx=0).stop 2024-08-18T13:47:21.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 shutdown_connections 2024-08-18T13:47:21.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcdd8060860 0x7fcdd8062d10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec0c6880 0x7fcdec0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcdec0aef90 0x7fcdec067d70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 --2- 172.21.15.118:0/3066966613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec0aebc0 0x7fcdec14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 >> 172.21.15.118:0/3066966613 conn(0x7fcdec0cbba0 msgr2=0x7fcdec158660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:21.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 shutdown_connections 2024-08-18T13:47:21.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.612+0000 7fcdf087b700 1 -- 172.21.15.118:0/3066966613 wait complete. 2024-08-18T13:47:21.615 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 26 2024-08-18T13:47:21.626 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-08-18T13:47:21.626 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-08-18T13:47:21.626 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:21.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.728+0000 7f5e047f9700 1 Processor -- start 2024-08-18T13:47:21.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.728+0000 7f5e047f9700 1 -- start start 2024-08-18T13:47:21.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e047f9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5dfc14d1c0 0x7f5dfc14d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e047f9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc0c6f90 con 0x7f5dfc0c5c80 2024-08-18T13:47:21.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e047f9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc0c6650 con 0x7f5dfc06c330 2024-08-18T13:47:21.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e047f9700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc06a6f0 con 0x7f5dfc14d1c0 2024-08-18T13:47:21.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e02595700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5dfc14d1c0 0x7f5dfc14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e02d96700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f5dfc0c5c80 0x7f5dfc1567a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:56908/0 (socket says 172.21.15.118:56908) 2024-08-18T13:47:21.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e02d96700 1 -- 172.21.15.118:0/3342223904 learned_addr learned my addr 172.21.15.118:0/3342223904 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:47:21.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126255970 0 0) 0x7f5dfc0c6f90 con 0x7f5dfc0c5c80 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de8003570 con 0x7f5dfc0c5c80 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567353045 0 0) 0x7f5dfc0c6650 con 0x7f5dfc06c330 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dfc0c6f90 con 0x7f5dfc06c330 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1718401379 0 0) 0x7f5de8003570 con 0x7f5dfc0c5c80 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 >> v2:172.21.15.118:3300/0 conn(0x7f5dfc14d1c0 msgr2=0x7f5dfc14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 --2- 172.21.15.118:0/3342223904 >> v2:172.21.15.118:3300/0 conn(0x7f5dfc14d1c0 0x7f5dfc14d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 >> v1:172.21.15.178:6789/0 conn(0x7f5dfc06c330 legacy=0x7f5dfc0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc06a900 con 0x7f5dfc0c5c80 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f5df40017f0 con 0x7f5dfc0c5c80 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 >> v1:172.21.15.118:6789/0 conn(0x7f5dfc0c5c80 legacy=0x7f5dfc1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5de8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5de8004810 0x7f5de8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5de8005230 0x7f5de800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc06a6f0 con 0x7f5de8003bf0 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de800c200 con 0x7f5de8005230 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de800c3c0 con 0x7f5de8004810 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e02595700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5de8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e01d94700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5de8004810 0x7f5de8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e02d96700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5de8005230 0x7f5de800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e02595700 1 -- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5de8004810 msgr2=0x7f5de8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.732+0000 7f5e02595700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5de8004810 0x7f5de8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e02595700 1 -- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5de8005230 msgr2=0x7f5de800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e02595700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5de8005230 0x7f5de800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.741 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e02595700 1 -- 172.21.15.118:0/3342223904 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8003570 con 0x7f5de8003bf0 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e02d96700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5de8005230 0x7f5de800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e02d96700 1 -- 172.21.15.118:0/3342223904 reap_dead start 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e02595700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5de8003fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5df800bb90 tx=0x7f5df8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce163c72f3b2cd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800d5d0 con 0x7f5de8003bf0 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df800d730 con 0x7f5de8003bf0 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e01593700 1 -- 172.21.15.118:0/3342223904 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800daf0 con 0x7f5de8003bf0 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e047f9700 1 -- 172.21.15.118:0/3342223904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 msgr2=0x7f5de8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5de8003fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5df800bb90 tx=0x7f5df8008040 comp rx=0 tx=0).stop 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e047f9700 1 -- 172.21.15.118:0/3342223904 shutdown_connections 2024-08-18T13:47:21.742 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3342223904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5de8003fe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e047f9700 1 -- 172.21.15.118:0/3342223904 >> 172.21.15.118:0/3342223904 conn(0x7f5dfc0ca7b0 msgr2=0x7f5dfc0c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:21.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e047f9700 1 -- 172.21.15.118:0/3342223904 shutdown_connections 2024-08-18T13:47:21.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.736+0000 7f5e047f9700 1 -- 172.21.15.118:0/3342223904 wait complete. 2024-08-18T13:47:21.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 Processor -- start 2024-08-18T13:47:21.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 -- start start 2024-08-18T13:47:21.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5dfc15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dfc0c8fa0 0x7f5dfc1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dfc166580 0x7f5dfc1669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc1508a0 con 0x7f5de8003bf0 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc1575f0 con 0x7f5dfc166580 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc166f10 con 0x7f5dfc0c8fa0 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5dfc15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5dfc15a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:32892/0 (socket says 172.21.15.118:32892) 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 -- 172.21.15.118:0/3212167325 learned_addr learned my addr 172.21.15.118:0/3212167325 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e01d94700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dfc0c8fa0 0x7f5dfc1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02d96700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dfc166580 0x7f5dfc1669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 -- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dfc0c8fa0 msgr2=0x7f5dfc1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dfc0c8fa0 0x7f5dfc1588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 -- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dfc166580 msgr2=0x7f5dfc1669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dfc166580 0x7f5dfc1669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 -- 172.21.15.118:0/3212167325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc156b70 con 0x7f5de8003bf0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02d96700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dfc166580 0x7f5dfc1669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e02595700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5dfc15a2b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b600 tx=0x7f5df801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5df27fc700 1 -- 172.21.15.118:0/3212167325 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8014810 con 0x7f5de8003bf0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5df27fc700 1 -- 172.21.15.118:0/3212167325 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df8014970 con 0x7f5de8003bf0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc156e50 con 0x7f5de8003bf0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5df27fc700 1 -- 172.21.15.118:0/3212167325 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8014d30 con 0x7f5de8003bf0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.740+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc150250 con 0x7f5de8003bf0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.744+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc0648f0 con 0x7f5de8003bf0 2024-08-18T13:47:21.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.744+0000 7f5df27fc700 1 -- 172.21.15.118:0/3212167325 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5df801d610 con 0x7f5de8003bf0 2024-08-18T13:47:21.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.744+0000 7f5df27fc700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5de0060890 0x7f5de0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:21.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.744+0000 7f5e01d94700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5de0060890 0x7f5de0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:21.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.744+0000 7f5df27fc700 1 -- 172.21.15.118:0/3212167325 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f5df807ea50 con 0x7f5de8003bf0 2024-08-18T13:47:21.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.744+0000 7f5e01d94700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5de0060890 0x7f5de0062d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc14ba80 tx=0x7f5dec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:21.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.748+0000 7f5df27fc700 1 -- 172.21.15.118:0/3212167325 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5df804e990 con 0x7f5de8003bf0 2024-08-18T13:47:21.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.968+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dfc0c6430 con 0x7f5de8003bf0 2024-08-18T13:47:21.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.968+0000 7f5df27fc700 1 -- 172.21.15.118:0/3212167325 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f5df802c020 con 0x7f5de8003bf0 2024-08-18T13:47:21.973 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:21.973 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:20.593666+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5de0060890 msgr2=0x7f5de0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5de0060890 0x7f5de0062d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc14ba80 tx=0x7f5dec009040 comp rx=0 tx=0).stop 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 msgr2=0x7f5dfc15a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5dfc15a2b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b600 tx=0x7f5df801b000 comp rx=0 tx=0).stop 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 shutdown_connections 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5de0060890 0x7f5de0062d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dfc166580 0x7f5dfc1669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dfc0c8fa0 0x7f5dfc1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 --2- 172.21.15.118:0/3212167325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5de8003bf0 0x7f5dfc15a2b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 >> 172.21.15.118:0/3212167325 conn(0x7f5dfc0ca7b0 msgr2=0x7f5dfc0c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:21.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 shutdown_connections 2024-08-18T13:47:21.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:21.972+0000 7f5e047f9700 1 -- 172.21.15.118:0/3212167325 wait complete. 2024-08-18T13:47:21.977 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 26 2024-08-18T13:47:21.987 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 26, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 26, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5350": { "addr": "172.21.15.178:6837/1054354354", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6836", "nonce": 1054354354, "type": "v2" }, { "addr": "172.21.15.178:6837", "nonce": 1054354354, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5350, "incarnation": 26, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:47:20.593666+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5350 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:47:23.990 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:24.122 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf320ad700 1 Processor -- start 2024-08-18T13:47:24.122 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf320ad700 1 -- start start 2024-08-18T13:47:24.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf320ad700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faf2c0c14f0 0x7faf2c05de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:24.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf320ad700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0677e0 con 0x7faf2c068380 2024-08-18T13:47:24.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf320ad700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0c2610 con 0x7faf2c0c1120 2024-08-18T13:47:24.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf320ad700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c0c2050 con 0x7faf2c0c14f0 2024-08-18T13:47:24.124 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf318ac700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faf2c0c14f0 0x7faf2c05de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:24.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf318ac700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7faf2c0c14f0 0x7faf2c05de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58298/0 (socket says 172.21.15.118:58298) 2024-08-18T13:47:24.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf318ac700 1 -- 172.21.15.118:0/1781343476 learned_addr learned my addr 172.21.15.118:0/1781343476 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:24.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf308aa700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7faf2c068380 0x7faf2c14bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55944/0 (socket says 172.21.15.118:55944) 2024-08-18T13:47:24.125 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf318ac700 1 -- 172.21.15.118:0/1781343476 >> v1:172.21.15.178:6789/0 conn(0x7faf2c0c1120 legacy=0x7faf2c067c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:24.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf318ac700 1 -- 172.21.15.118:0/1781343476 >> v1:172.21.15.118:6789/0 conn(0x7faf2c068380 legacy=0x7faf2c14bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:24.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf23fff700 1 -- 172.21.15.118:0/1781343476 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124368484 0 0) 0x7faf2c0677e0 con 0x7faf2c068380 2024-08-18T13:47:24.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.120+0000 7faf318ac700 1 -- 172.21.15.118:0/1781343476 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0c2330 con 0x7faf2c0c14f0 2024-08-18T13:47:24.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf318ac700 1 --2- 172.21.15.118:0/1781343476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c0c14f0 0x7faf2c05de20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0099b0 tx=0x7faf1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b14211e80e5d1b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:24.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf23fff700 1 -- 172.21.15.118:0/1781343476 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00b860 con 0x7faf2c0c14f0 2024-08-18T13:47:24.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf23fff700 1 -- 172.21.15.118:0/1781343476 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c00b9c0 con 0x7faf2c0c14f0 2024-08-18T13:47:24.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf23fff700 1 -- 172.21.15.118:0/1781343476 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00bd60 con 0x7faf2c0c14f0 2024-08-18T13:47:24.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 -- 172.21.15.118:0/1781343476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c0c14f0 msgr2=0x7faf2c05de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:24.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 --2- 172.21.15.118:0/1781343476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c0c14f0 0x7faf2c05de20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0099b0 tx=0x7faf1c009250 comp rx=0 tx=0).stop 2024-08-18T13:47:24.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 -- 172.21.15.118:0/1781343476 shutdown_connections 2024-08-18T13:47:24.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 --2- 172.21.15.118:0/1781343476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c0c14f0 0x7faf2c05de20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 -- 172.21.15.118:0/1781343476 >> 172.21.15.118:0/1781343476 conn(0x7faf2c06bdf0 msgr2=0x7faf2c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:24.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 -- 172.21.15.118:0/1781343476 shutdown_connections 2024-08-18T13:47:24.127 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 -- 172.21.15.118:0/1781343476 wait complete. 2024-08-18T13:47:24.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 Processor -- start 2024-08-18T13:47:24.128 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.124+0000 7faf320ad700 1 -- start start 2024-08-18T13:47:24.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c068380 0x7faf2c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:24.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf2c0c1120 0x7faf2c149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:24.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf2c0c14f0 0x7faf2c1650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c165de0 con 0x7faf2c068380 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c1438b0 con 0x7faf2c0c1120 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c14b870 con 0x7faf2c0c14f0 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c068380 0x7faf2c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c068380 0x7faf2c149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58312/0 (socket says 172.21.15.118:58312) 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 -- 172.21.15.118:0/1916266188 learned_addr learned my addr 172.21.15.118:0/1916266188 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf308aa700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf2c0c1120 0x7faf2c149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf318ac700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf2c0c14f0 0x7faf2c1650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 -- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf2c0c14f0 msgr2=0x7faf2c1650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf2c0c14f0 0x7faf2c1650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 -- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf2c0c1120 msgr2=0x7faf2c149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf2c0c1120 0x7faf2c149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 -- 172.21.15.118:0/1916266188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c0094f0 con 0x7faf2c068380 2024-08-18T13:47:24.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf308aa700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf2c0c1120 0x7faf2c149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:47:24.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf310ab700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c068380 0x7faf2c149920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf2400cb10 tx=0x7faf240017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:24.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf217fa700 1 -- 172.21.15.118:0/1916266188 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24006ba0 con 0x7faf2c068380 2024-08-18T13:47:24.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf217fa700 1 -- 172.21.15.118:0/1916266188 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf24006d00 con 0x7faf2c068380 2024-08-18T13:47:24.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c142ca0 con 0x7faf2c068380 2024-08-18T13:47:24.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf217fa700 1 -- 172.21.15.118:0/1916266188 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2400a9b0 con 0x7faf2c068380 2024-08-18T13:47:24.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.128+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c143260 con 0x7faf2c068380 2024-08-18T13:47:24.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.132+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c0713a0 con 0x7faf2c068380 2024-08-18T13:47:24.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.132+0000 7faf217fa700 1 -- 172.21.15.118:0/1916266188 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7faf24012420 con 0x7faf2c068380 2024-08-18T13:47:24.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.132+0000 7faf217fa700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faf10060860 0x7faf10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:24.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.132+0000 7faf308aa700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faf10060860 0x7faf10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:24.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.132+0000 7faf217fa700 1 -- 172.21.15.118:0/1916266188 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7faf2407cfe0 con 0x7faf2c068380 2024-08-18T13:47:24.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.132+0000 7faf308aa700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faf10060860 0x7faf10062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faf2c05ee10 tx=0x7faf28007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:24.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.140+0000 7faf217fa700 1 -- 172.21.15.118:0/1916266188 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faf2404cf20 con 0x7faf2c068380 2024-08-18T13:47:24.356 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.352+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf2c064cf0 con 0x7faf2c068380 2024-08-18T13:47:24.356 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.352+0000 7faf217fa700 1 -- 172.21.15.118:0/1916266188 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5369 (secure 0 0 0) 0x7faf24050540 con 0x7faf2c068380 2024-08-18T13:47:24.356 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:24.357 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:20.593666+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:24.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faf10060860 msgr2=0x7faf10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:24.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faf10060860 0x7faf10062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faf2c05ee10 tx=0x7faf28007c90 comp rx=0 tx=0).stop 2024-08-18T13:47:24.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c068380 msgr2=0x7faf2c149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:24.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c068380 0x7faf2c149920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf2400cb10 tx=0x7faf240017b0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 shutdown_connections 2024-08-18T13:47:24.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faf10060860 0x7faf10062d10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf2c0c14f0 0x7faf2c1650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf2c0c1120 0x7faf2c149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 --2- 172.21.15.118:0/1916266188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faf2c068380 0x7faf2c149920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:24.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 >> 172.21.15.118:0/1916266188 conn(0x7faf2c06bdf0 msgr2=0x7faf2c0c3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:24.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 shutdown_connections 2024-08-18T13:47:24.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:24.356+0000 7faf320ad700 1 -- 172.21.15.118:0/1916266188 wait complete. 2024-08-18T13:47:24.360 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 26 2024-08-18T13:47:24.372 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:47:26.372 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:26.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 Processor -- start 2024-08-18T13:47:26.490 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 -- start start 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb0c8156810 0x7fb0c80c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8157060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c80ca970 con 0x7fb0c806ba70 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c81575a0 con 0x7fb0c8156810 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80becb0 con 0x7fb0c8156be0 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb0c8156810 0x7fb0c80c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb0c8156810 0x7fb0c80c07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58324/0 (socket says 172.21.15.118:58324) 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 -- 172.21.15.118:0/3846115195 learned_addr learned my addr 172.21.15.118:0/3846115195 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:26.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c7fff700 1 --2- 172.21.15.118:0/3846115195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 -- 172.21.15.118:0/3846115195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 msgr2=0x7fb0c8157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 --2- 172.21.15.118:0/3846115195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c6ffd700 1 -- 172.21.15.118:0/3846115195 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817605550 0 0) 0x7fb0c80ca970 con 0x7fb0c806ba70 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 -- 172.21.15.118:0/3846115195 >> v1:172.21.15.178:6789/0 conn(0x7fb0c806ba70 legacy=0x7fb0c806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 -- 172.21.15.118:0/3846115195 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80bef90 con 0x7fb0c8156810 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c77fe700 1 --2- 172.21.15.118:0/3846115195 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c80c07d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80099b0 tx=0x7fb0b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917cd9e6719a50e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c6ffd700 1 -- 172.21.15.118:0/3846115195 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8009ea0 con 0x7fb0c8156810 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c6ffd700 1 -- 172.21.15.118:0/3846115195 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b800b860 con 0x7fb0c8156810 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0c6ffd700 1 -- 172.21.15.118:0/3846115195 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800bc00 con 0x7fb0c8156810 2024-08-18T13:47:26.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 -- 172.21.15.118:0/3846115195 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 msgr2=0x7fb0c80c07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:26.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.488+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/3846115195 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c80c07d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80099b0 tx=0x7fb0b8009250 comp rx=0 tx=0).stop 2024-08-18T13:47:26.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- 172.21.15.118:0/3846115195 shutdown_connections 2024-08-18T13:47:26.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/3846115195 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/3846115195 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c80c07d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- 172.21.15.118:0/3846115195 >> 172.21.15.118:0/3846115195 conn(0x7fb0c806e090 msgr2=0x7fb0c80c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:26.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- 172.21.15.118:0/3846115195 shutdown_connections 2024-08-18T13:47:26.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- 172.21.15.118:0/3846115195 wait complete. 2024-08-18T13:47:26.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 Processor -- start 2024-08-18T13:47:26.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- start start 2024-08-18T13:47:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0c806ba70 0x7fb0c80684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c8149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8165510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c8068b90 con 0x7fb0c8156810 2024-08-18T13:47:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c8165080 con 0x7fb0c806ba70 2024-08-18T13:47:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0ccbce700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80c4410 con 0x7fb0c8156be0 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0bffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c8149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0bffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c8149c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58326/0 (socket says 172.21.15.118:58326) 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.492+0000 7fb0c77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0c806ba70 0x7fb0c80684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bffff700 1 -- 172.21.15.118:0/2254695022 learned_addr learned my addr 172.21.15.118:0/2254695022 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0c7fff700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8165510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bffff700 1 -- 172.21.15.118:0/2254695022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 msgr2=0x7fb0c8165510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bffff700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8165510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bffff700 1 -- 172.21.15.118:0/2254695022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0c806ba70 msgr2=0x7fb0c80684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bffff700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0c806ba70 0x7fb0c80684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bffff700 1 -- 172.21.15.118:0/2254695022 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80094f0 con 0x7fb0c8156810 2024-08-18T13:47:26.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bffff700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c8149c60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c530 tx=0x7fb0b000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:26.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bf7fe700 1 -- 172.21.15.118:0/2254695022 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0011590 con 0x7fb0c8156810 2024-08-18T13:47:26.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bf7fe700 1 -- 172.21.15.118:0/2254695022 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b0011710 con 0x7fb0c8156810 2024-08-18T13:47:26.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c80c46f0 con 0x7fb0c8156810 2024-08-18T13:47:26.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bf7fe700 1 -- 172.21.15.118:0/2254695022 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0011ad0 con 0x7fb0c8156810 2024-08-18T13:47:26.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c80c4cb0 con 0x7fb0c8156810 2024-08-18T13:47:26.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c80c8880 con 0x7fb0c8156810 2024-08-18T13:47:26.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bf7fe700 1 -- 172.21.15.118:0/2254695022 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb0b001b3b0 con 0x7fb0c8156810 2024-08-18T13:47:26.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bf7fe700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb0a8060890 0x7fb0a8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:26.501 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0bf7fe700 1 -- 172.21.15.118:0/2254695022 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7fb0b007c4b0 con 0x7fb0c8156810 2024-08-18T13:47:26.501 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.496+0000 7fb0c77fe700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb0a8060890 0x7fb0a8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:26.501 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.500+0000 7fb0c77fe700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb0a8060890 0x7fb0a8062d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80099b0 tx=0x7fb0b8017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:26.507 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.504+0000 7fb0bf7fe700 1 -- 172.21.15.118:0/2254695022 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb0c80c8880 con 0x7fb0c8156810 2024-08-18T13:47:26.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.720+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0c80c8880 con 0x7fb0c8156810 2024-08-18T13:47:26.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.720+0000 7fb0bf7fe700 1 -- 172.21.15.118:0/2254695022 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5369 (secure 0 0 0) 0x7fb0c80c8880 con 0x7fb0c8156810 2024-08-18T13:47:26.725 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:26.725 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:20.593666+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb0a8060890 msgr2=0x7fb0a8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb0a8060890 0x7fb0a8062d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80099b0 tx=0x7fb0b8017000 comp rx=0 tx=0).stop 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 msgr2=0x7fb0c8149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c8149c60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c530 tx=0x7fb0b000c840 comp rx=0 tx=0).stop 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 shutdown_connections 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb0a8060890 0x7fb0a8062d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0c8156be0 0x7fb0c8165510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb0c8156810 0x7fb0c8149c60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 --2- 172.21.15.118:0/2254695022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0c806ba70 0x7fb0c80684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:26.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 >> 172.21.15.118:0/2254695022 conn(0x7fb0c806e090 msgr2=0x7fb0c814b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:26.728 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 shutdown_connections 2024-08-18T13:47:26.728 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:26.724+0000 7fb0ccbce700 1 -- 172.21.15.118:0/2254695022 wait complete. 2024-08-18T13:47:26.728 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 26 2024-08-18T13:47:26.737 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:47:28.738 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:28.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 Processor -- start 2024-08-18T13:47:28.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 -- start start 2024-08-18T13:47:28.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:28.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f189414f7e0 0x7f18940abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:28.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1894158830 0x7f18940c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:28.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18940c1c80 con 0x7f1894158830 2024-08-18T13:47:28.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18940c18e0 con 0x7f189414f7e0 2024-08-18T13:47:28.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1899cf5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c09f0 con 0x7f189414f410 2024-08-18T13:47:28.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1894158830 0x7f18940c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:28.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1894158830 0x7f18940c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58338/0 (socket says 172.21.15.118:58338) 2024-08-18T13:47:28.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 -- 172.21.15.118:0/32200220 learned_addr learned my addr 172.21.15.118:0/32200220 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:28.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1898cf3700 1 --2- 172.21.15.118:0/32200220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:28.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1893fff700 1 --2- 172.21.15.118:0/32200220 >> v2:172.21.15.178:3300/0 conn(0x7f189414f7e0 0x7f18940abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:28.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 -- 172.21.15.118:0/32200220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 msgr2=0x7f18940ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:28.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 --2- 172.21.15.118:0/32200220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 -- 172.21.15.118:0/32200220 >> v2:172.21.15.178:3300/0 conn(0x7f189414f7e0 msgr2=0x7f18940abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:28.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 --2- 172.21.15.118:0/32200220 >> v2:172.21.15.178:3300/0 conn(0x7f189414f7e0 0x7f18940abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 -- 172.21.15.118:0/32200220 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940c0cd0 con 0x7f1894158830 2024-08-18T13:47:28.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f1898cf3700 1 --2- 172.21.15.118:0/32200220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:47:28.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.848+0000 7f18994f4700 1 --2- 172.21.15.118:0/32200220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f18940c7530 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f188c014ac0 tx=0x7f188c0143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e3f651f8f3e422 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:28.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f18937fe700 1 -- 172.21.15.118:0/32200220 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c013c20 con 0x7f1894158830 2024-08-18T13:47:28.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f18937fe700 1 -- 172.21.15.118:0/32200220 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c013d80 con 0x7f1894158830 2024-08-18T13:47:28.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f18937fe700 1 -- 172.21.15.118:0/32200220 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c014120 con 0x7f1894158830 2024-08-18T13:47:28.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- 172.21.15.118:0/32200220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 msgr2=0x7f18940c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:28.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 --2- 172.21.15.118:0/32200220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f18940c7530 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f188c014ac0 tx=0x7f188c0143c0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- 172.21.15.118:0/32200220 shutdown_connections 2024-08-18T13:47:28.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 --2- 172.21.15.118:0/32200220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f18940c7530 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 --2- 172.21.15.118:0/32200220 >> v2:172.21.15.178:3300/0 conn(0x7f189414f7e0 0x7f18940abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 --2- 172.21.15.118:0/32200220 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- 172.21.15.118:0/32200220 >> 172.21.15.118:0/32200220 conn(0x7f18940c8620 msgr2=0x7f18940c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:28.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- 172.21.15.118:0/32200220 shutdown_connections 2024-08-18T13:47:28.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- 172.21.15.118:0/32200220 wait complete. 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 Processor -- start 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- start start 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189414f7e0 0x7f18941578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f1894157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894143b20 con 0x7f1894158830 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940c8440 con 0x7f189414f7e0 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1899cf5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1894161160 con 0x7f189414f410 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f18994f4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f1894157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f18994f4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f1894157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58346/0 (socket says 172.21.15.118:58346) 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f18994f4700 1 -- 172.21.15.118:0/2258718435 learned_addr learned my addr 172.21.15.118:0/2258718435 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1898cf3700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:28.863 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.852+0000 7f1893fff700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189414f7e0 0x7f18941578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f18994f4700 1 -- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 msgr2=0x7f18940c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f18994f4700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f18994f4700 1 -- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189414f7e0 msgr2=0x7f18941578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f18994f4700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189414f7e0 0x7f18941578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f18994f4700 1 -- 172.21.15.118:0/2258718435 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c014600 con 0x7f1894158830 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f18994f4700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f1894157e00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f188c014fd0 tx=0x7f188c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1890ff9700 1 -- 172.21.15.118:0/2258718435 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0032e0 con 0x7f1894158830 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1890ff9700 1 -- 172.21.15.118:0/2258718435 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c011b10 con 0x7f1894158830 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1894161440 con 0x7f1894158830 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1890ff9700 1 -- 172.21.15.118:0/2258718435 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c026870 con 0x7f1894158830 2024-08-18T13:47:28.864 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1894161a00 con 0x7f1894158830 2024-08-18T13:47:28.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1890ff9700 1 -- 172.21.15.118:0/2258718435 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f188c003030 con 0x7f1894158830 2024-08-18T13:47:28.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18940713a0 con 0x7f1894158830 2024-08-18T13:47:28.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1890ff9700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1878060890 0x7f1878062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:28.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1890ff9700 1 -- 172.21.15.118:0/2258718435 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f188c087760 con 0x7f1894158830 2024-08-18T13:47:28.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1898cf3700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1878060890 0x7f1878062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:28.865 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.856+0000 7f1898cf3700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1878060890 0x7f1878062d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1888006f00 tx=0x7f1888008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:28.867 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:28.864+0000 7f1890ff9700 1 -- 172.21.15.118:0/2258718435 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f188c05b130 con 0x7f1894158830 2024-08-18T13:47:29.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.092+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18940c29a0 con 0x7f1894158830 2024-08-18T13:47:29.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.092+0000 7f1890ff9700 1 -- 172.21.15.118:0/2258718435 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5373 (secure 0 0 0) 0x7f188c05acc0 con 0x7f1894158830 2024-08-18T13:47:29.095 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:29.095 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:28.696670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1878060890 msgr2=0x7f1878062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1878060890 0x7f1878062d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1888006f00 tx=0x7f1888008040 comp rx=0 tx=0).stop 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 msgr2=0x7f1894157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f1894157e00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f188c014fd0 tx=0x7f188c01f000 comp rx=0 tx=0).stop 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 shutdown_connections 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1878060890 0x7f1878062d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1894158830 0x7f1894157e00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f189414f7e0 0x7f18941578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 --2- 172.21.15.118:0/2258718435 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f189414f410 0x7f18940c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:29.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 >> 172.21.15.118:0/2258718435 conn(0x7f18940c8620 msgr2=0x7f1894158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:29.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 shutdown_connections 2024-08-18T13:47:29.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:29.096+0000 7f1899cf5700 1 -- 172.21.15.118:0/2258718435 wait complete. 2024-08-18T13:47:29.099 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 27 2024-08-18T13:47:29.109 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:47:31.110 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:31.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba9ec700 1 Processor -- start 2024-08-18T13:47:31.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba9ec700 1 -- start start 2024-08-18T13:47:31.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba9ec700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b4156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:31.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba9ec700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f36b4067d50 0x7f36b4153980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:31.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba9ec700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b41534f0 con 0x7f36b40c4320 2024-08-18T13:47:31.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba9ec700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b41574c0 con 0x7f36b4067d50 2024-08-18T13:47:31.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba9ec700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4152f70 con 0x7f36b40c46f0 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba1eb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f36b4067d50 0x7f36b4153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36b99ea700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f36b40c4320 0x7f36b405e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55954/0 (socket says 172.21.15.118:55954) 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36b99ea700 1 -- 172.21.15.118:0/2233934150 learned_addr learned my addr 172.21.15.118:0/2233934150 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36b91e9700 1 --2- 172.21.15.118:0/2233934150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba1eb700 1 -- 172.21.15.118:0/2233934150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 msgr2=0x7f36b4156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba1eb700 1 --2- 172.21.15.118:0/2233934150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b4156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba1eb700 1 -- 172.21.15.118:0/2233934150 >> v1:172.21.15.118:6789/0 conn(0x7f36b40c4320 legacy=0x7f36b405e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:31.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba1eb700 1 -- 172.21.15.118:0/2233934150 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4156c20 con 0x7f36b4067d50 2024-08-18T13:47:31.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.220+0000 7f36ba1eb700 1 --2- 172.21.15.118:0/2233934150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b4153980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009e60 tx=0x7f36ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8404b70b930a1ec9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:31.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36b89e8700 1 -- 172.21.15.118:0/2233934150 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00e070 con 0x7f36b4067d50 2024-08-18T13:47:31.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36b89e8700 1 -- 172.21.15.118:0/2233934150 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00bbf0 con 0x7f36b4067d50 2024-08-18T13:47:31.226 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36b89e8700 1 -- 172.21.15.118:0/2233934150 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac012430 con 0x7f36b4067d50 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 -- 172.21.15.118:0/2233934150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 msgr2=0x7f36b4153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/2233934150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b4153980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009e60 tx=0x7f36ac009780 comp rx=0 tx=0).stop 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 -- 172.21.15.118:0/2233934150 shutdown_connections 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/2233934150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b4153980 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/2233934150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b4156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 -- 172.21.15.118:0/2233934150 >> 172.21.15.118:0/2233934150 conn(0x7f36b40c81e0 msgr2=0x7f36b40c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 -- 172.21.15.118:0/2233934150 shutdown_connections 2024-08-18T13:47:31.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 -- 172.21.15.118:0/2233934150 wait complete. 2024-08-18T13:47:31.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 Processor -- start 2024-08-18T13:47:31.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.224+0000 7f36ba9ec700 1 -- start start 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b41654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b40c4320 0x7f36b4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b40c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b406b170 con 0x7f36b4067d50 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4151750 con 0x7f36b40c4320 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40bf7d0 con 0x7f36b40c46f0 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b41654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:31.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b91e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b40c4320 0x7f36b4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b41654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58362/0 (socket says 172.21.15.118:58362) 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 -- 172.21.15.118:0/1898376481 learned_addr learned my addr 172.21.15.118:0/1898376481 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba1eb700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b40c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 -- 172.21.15.118:0/1898376481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 msgr2=0x7f36b40c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b40c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 -- 172.21.15.118:0/1898376481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b40c4320 msgr2=0x7f36b4154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b40c4320 0x7f36b4154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 -- 172.21.15.118:0/1898376481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac009a20 con 0x7f36b4067d50 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36b99ea700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b41654c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c7c0 tx=0x7f36b0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36a9ffb700 1 -- 172.21.15.118:0/1898376481 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0005b30 con 0x7f36b4067d50 2024-08-18T13:47:31.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36a9ffb700 1 -- 172.21.15.118:0/1898376481 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b0005c90 con 0x7f36b4067d50 2024-08-18T13:47:31.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36a9ffb700 1 -- 172.21.15.118:0/1898376481 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0004a80 con 0x7f36b4067d50 2024-08-18T13:47:31.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b4165a00 con 0x7f36b4067d50 2024-08-18T13:47:31.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b40bfa30 con 0x7f36b4067d50 2024-08-18T13:47:31.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.228+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b40713a0 con 0x7f36b4067d50 2024-08-18T13:47:31.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.232+0000 7f36a9ffb700 1 -- 172.21.15.118:0/1898376481 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f36b00070d0 con 0x7f36b4067d50 2024-08-18T13:47:31.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.232+0000 7f36a9ffb700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3698060890 0x7f3698062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:31.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.232+0000 7f36a9ffb700 1 -- 172.21.15.118:0/1898376481 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f36b007c3c0 con 0x7f36b4067d50 2024-08-18T13:47:31.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.232+0000 7f36b91e9700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3698060890 0x7f3698062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:31.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.232+0000 7f36b91e9700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3698060890 0x7f3698062d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f36b40bf600 tx=0x7f36a40091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:31.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.240+0000 7f36a9ffb700 1 -- 172.21.15.118:0/1898376481 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f36b004a480 con 0x7f36b4067d50 2024-08-18T13:47:31.458 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.456+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36b40c4b70 con 0x7f36b4067d50 2024-08-18T13:47:31.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.456+0000 7f36a9ffb700 1 -- 172.21.15.118:0/1898376481 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5373 (secure 0 0 0) 0x7f36b004c300 con 0x7f36b4067d50 2024-08-18T13:47:31.459 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:31.459 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:28.696670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:31.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3698060890 msgr2=0x7f3698062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:31.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3698060890 0x7f3698062d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f36b40bf600 tx=0x7f36a40091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 msgr2=0x7f36b41654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:31.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b41654c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36b000c7c0 tx=0x7f36b0007f70 comp rx=0 tx=0).stop 2024-08-18T13:47:31.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 shutdown_connections 2024-08-18T13:47:31.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3698060890 0x7f3698062d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36b40c46f0 0x7f36b40c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b40c4320 0x7f36b4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 --2- 172.21.15.118:0/1898376481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36b4067d50 0x7f36b41654c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:31.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 >> 172.21.15.118:0/1898376481 conn(0x7f36b40c81e0 msgr2=0x7f36b406a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:31.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 shutdown_connections 2024-08-18T13:47:31.466 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:31.460+0000 7f36ba9ec700 1 -- 172.21.15.118:0/1898376481 wait complete. 2024-08-18T13:47:31.467 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 27 2024-08-18T13:47:31.477 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:47:33.478 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:33.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f280ce700 1 Processor -- start 2024-08-18T13:47:33.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f280ce700 1 -- start start 2024-08-18T13:47:33.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f280ce700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5f20155850 0x7f5f200c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:33.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f280ce700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5f20156330 0x7f5f201520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:33.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f280ce700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200687b0 con 0x7f5f200c12a0 2024-08-18T13:47:33.602 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f280ce700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f20068c40 con 0x7f5f20155850 2024-08-18T13:47:33.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f280ce700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2005e9f0 con 0x7f5f20156330 2024-08-18T13:47:33.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f25e6a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5f20155850 0x7f5f200c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:33.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f25669700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f5f200c12a0 0x7f5f20155c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:55960/0 (socket says 172.21.15.118:55960) 2024-08-18T13:47:33.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.596+0000 7f5f25669700 1 -- 172.21.15.118:0/1411511198 learned_addr learned my addr 172.21.15.118:0/1411511198 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f2666b700 1 --2- 172.21.15.118:0/1411511198 >> v2:172.21.15.178:3300/0 conn(0x7f5f20156330 0x7f5f201520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f24e68700 1 -- 172.21.15.118:0/1411511198 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024419303 0 0) 0x7f5f200687b0 con 0x7f5f200c12a0 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f24e68700 1 -- 172.21.15.118:0/1411511198 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f0c003570 con 0x7f5f200c12a0 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f25e6a700 1 -- 172.21.15.118:0/1411511198 >> v2:172.21.15.178:3300/0 conn(0x7f5f20156330 msgr2=0x7f5f201520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f25e6a700 1 --2- 172.21.15.118:0/1411511198 >> v2:172.21.15.178:3300/0 conn(0x7f5f20156330 0x7f5f201520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f25e6a700 1 -- 172.21.15.118:0/1411511198 >> v1:172.21.15.118:6789/0 conn(0x7f5f200c12a0 legacy=0x7f5f20155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f25e6a700 1 -- 172.21.15.118:0/1411511198 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20067e80 con 0x7f5f20155850 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f2666b700 1 --2- 172.21.15.118:0/1411511198 >> v2:172.21.15.178:3300/0 conn(0x7f5f20156330 0x7f5f201520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f25e6a700 1 --2- 172.21.15.118:0/1411511198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f20155850 0x7f5f200c0d60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5f20067d90 tx=0x7f5f1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f242ef644d215 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:33.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f24e68700 1 -- 172.21.15.118:0/1411511198 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00b690 con 0x7f5f20155850 2024-08-18T13:47:33.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f24e68700 1 -- 172.21.15.118:0/1411511198 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00b7f0 con 0x7f5f20155850 2024-08-18T13:47:33.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f24e68700 1 -- 172.21.15.118:0/1411511198 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00bb70 con 0x7f5f20155850 2024-08-18T13:47:33.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 -- 172.21.15.118:0/1411511198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f20155850 msgr2=0x7f5f200c0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:33.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 --2- 172.21.15.118:0/1411511198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f20155850 0x7f5f200c0d60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5f20067d90 tx=0x7f5f1c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 -- 172.21.15.118:0/1411511198 shutdown_connections 2024-08-18T13:47:33.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 --2- 172.21.15.118:0/1411511198 >> v2:172.21.15.178:3300/0 conn(0x7f5f20156330 0x7f5f201520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 --2- 172.21.15.118:0/1411511198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f20155850 0x7f5f200c0d60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 -- 172.21.15.118:0/1411511198 >> 172.21.15.118:0/1411511198 conn(0x7f5f200c8e70 msgr2=0x7f5f200c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 -- 172.21.15.118:0/1411511198 shutdown_connections 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.600+0000 7f5f280ce700 1 -- 172.21.15.118:0/1411511198 wait complete. 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 Processor -- start 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 -- start start 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f200c12a0 0x7f5f20149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f20155850 0x7f5f20156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:33.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f200c3dd0 0x7f5f200c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200c7d20 con 0x7f5f200c3dd0 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20149990 con 0x7f5f20155850 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f201575d0 con 0x7f5f200c12a0 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f200c3dd0 0x7f5f200c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f200c3dd0 0x7f5f200c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:58388/0 (socket says 172.21.15.118:58388) 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 -- 172.21.15.118:0/2873758276 learned_addr learned my addr 172.21.15.118:0/2873758276 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f25669700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f20155850 0x7f5f20156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f25e6a700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f200c12a0 0x7f5f20149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:33.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 -- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f200c12a0 msgr2=0x7f5f20149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f200c12a0 0x7f5f20149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 -- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f20155850 msgr2=0x7f5f20156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f20155850 0x7f5f20156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 -- 172.21.15.118:0/2873758276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c009420 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f2666b700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f200c3dd0 0x7f5f200c61f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800a380 tx=0x7f5f18007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f15ffb700 1 -- 172.21.15.118:0/2873758276 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18015440 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f15ffb700 1 -- 172.21.15.118:0/2873758276 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f180155a0 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f200c7590 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f15ffb700 1 -- 172.21.15.118:0/2873758276 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18015960 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.604+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f20165130 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.608+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f200c3c20 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.608+0000 7f5f15ffb700 1 -- 172.21.15.118:0/2873758276 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5f1800d440 con 0x7f5f200c3dd0 2024-08-18T13:47:33.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.608+0000 7f5f15ffb700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f04060860 0x7f5f04062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:33.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.608+0000 7f5f15ffb700 1 -- 172.21.15.118:0/2873758276 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f5f1807dbd0 con 0x7f5f200c3dd0 2024-08-18T13:47:33.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.608+0000 7f5f25e6a700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f04060860 0x7f5f04062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:33.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.608+0000 7f5f25e6a700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f04060860 0x7f5f04062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0093f0 tx=0x7f5f1c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:33.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.612+0000 7f5f15ffb700 1 -- 172.21.15.118:0/2873758276 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5f1804db10 con 0x7f5f200c3dd0 2024-08-18T13:47:33.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.836+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f200648f0 con 0x7f5f200c3dd0 2024-08-18T13:47:33.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f15ffb700 1 -- 172.21.15.118:0/2873758276 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f5f1801a090 con 0x7f5f200c3dd0 2024-08-18T13:47:33.842 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:33.842 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:33.723760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:33.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f04060860 msgr2=0x7f5f04062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:33.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f04060860 0x7f5f04062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0093f0 tx=0x7f5f1c017000 comp rx=0 tx=0).stop 2024-08-18T13:47:33.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f200c3dd0 msgr2=0x7f5f200c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f200c3dd0 0x7f5f200c61f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800a380 tx=0x7f5f18007900 comp rx=0 tx=0).stop 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 shutdown_connections 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5f04060860 0x7f5f04062d10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5f200c3dd0 0x7f5f200c61f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f20155850 0x7f5f20156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 --2- 172.21.15.118:0/2873758276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f200c12a0 0x7f5f20149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 >> 172.21.15.118:0/2873758276 conn(0x7f5f200c8e70 msgr2=0x7f5f20155c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 shutdown_connections 2024-08-18T13:47:33.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:33.840+0000 7f5f280ce700 1 -- 172.21.15.118:0/2873758276 wait complete. 2024-08-18T13:47:33.846 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 28 2024-08-18T13:47:33.857 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 28, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 28, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5350": { "addr": "172.21.15.178:6837/1054354354", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6836", "nonce": 1054354354, "type": "v2" }, { "addr": "172.21.15.178:6837", "nonce": 1054354354, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5350, "incarnation": 26, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:rejoin", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:47:33.723760+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5350 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:47:35.858 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:35.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.956+0000 7f5fdc8b2700 1 Processor -- start 2024-08-18T13:47:35.959 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.956+0000 7f5fdc8b2700 1 -- start start 2024-08-18T13:47:35.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.956+0000 7f5fdc8b2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:35.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5fd80aef90 0x7f5fd80abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:35.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5fd80c6880 0x7f5fd80c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:35.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd8155140 con 0x7f5fd80c6880 2024-08-18T13:47:35.961 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd80af660 con 0x7f5fd80aef90 2024-08-18T13:47:35.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8155330 con 0x7f5fd80aebc0 2024-08-18T13:47:35.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5fd80c6880 0x7f5fd80c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:35.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5fd80c6880 0x7f5fd80c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:49498/0 (socket says 172.21.15.118:49498) 2024-08-18T13:47:35.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 -- 172.21.15.118:0/457093116 learned_addr learned my addr 172.21.15.118:0/457093116 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:35.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fcffff700 1 --2- 172.21.15.118:0/457093116 >> v2:172.21.15.178:3300/0 conn(0x7f5fd80aef90 0x7f5fd80abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:35.962 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd77fe700 1 --2- 172.21.15.118:0/457093116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:35.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 -- 172.21.15.118:0/457093116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 msgr2=0x7f5fd80b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:35.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 --2- 172.21.15.118:0/457093116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:35.963 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 -- 172.21.15.118:0/457093116 >> v2:172.21.15.178:3300/0 conn(0x7f5fd80aef90 msgr2=0x7f5fd80abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 --2- 172.21.15.118:0/457093116 >> v2:172.21.15.178:3300/0 conn(0x7f5fd80aef90 0x7f5fd80abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 -- 172.21.15.118:0/457093116 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd81555d0 con 0x7f5fd80c6880 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd7fff700 1 --2- 172.21.15.118:0/457093116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd80c6cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8070290 tx=0x7f5fd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33433a643888ac3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd6ffd700 1 -- 172.21.15.118:0/457093116 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000e070 con 0x7f5fd80c6880 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd6ffd700 1 -- 172.21.15.118:0/457093116 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd000b970 con 0x7f5fd80c6880 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fd6ffd700 1 -- 172.21.15.118:0/457093116 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00092e0 con 0x7f5fd80c6880 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/457093116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 msgr2=0x7f5fd80c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/457093116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd80c6cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8070290 tx=0x7f5fd0009840 comp rx=0 tx=0).stop 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/457093116 shutdown_connections 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/457093116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd80c6cc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/457093116 >> v2:172.21.15.178:3300/0 conn(0x7f5fd80aef90 0x7f5fd80abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/457093116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/457093116 >> 172.21.15.118:0/457093116 conn(0x7f5fd80cbba0 msgr2=0x7f5fd80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:35.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/457093116 shutdown_connections 2024-08-18T13:47:35.965 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.960+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/457093116 wait complete. 2024-08-18T13:47:35.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 Processor -- start 2024-08-18T13:47:35.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 -- start start 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd80aef90 0x7f5fd8157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80ab170 con 0x7f5fd80c6880 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80a7470 con 0x7f5fd80aef90 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8164d00 con 0x7f5fd80aebc0 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd814d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:49508/0 (socket says 172.21.15.118:49508) 2024-08-18T13:47:35.968 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 -- 172.21.15.118:0/1357764739 learned_addr learned my addr 172.21.15.118:0/1357764739 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:35.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd77fe700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:35.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fcffff700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd80aef90 0x7f5fd8157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:35.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 -- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 msgr2=0x7f5fd80a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:35.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:35.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 -- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd80aef90 msgr2=0x7f5fd8157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:35.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd80aef90 0x7f5fd8157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:35.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 -- 172.21.15.118:0/1357764739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0009ae0 con 0x7f5fd80c6880 2024-08-18T13:47:35.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fd7fff700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd814d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009880 tx=0x7f5fd0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:35.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fcf7fe700 1 -- 172.21.15.118:0/1357764739 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0008b10 con 0x7f5fd80c6880 2024-08-18T13:47:35.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fcf7fe700 1 -- 172.21.15.118:0/1357764739 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0008c70 con 0x7f5fd80c6880 2024-08-18T13:47:35.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd8164fe0 con 0x7f5fd80c6880 2024-08-18T13:47:35.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fcf7fe700 1 -- 172.21.15.118:0/1357764739 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0025460 con 0x7f5fd80c6880 2024-08-18T13:47:35.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.964+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd8165370 con 0x7f5fd80c6880 2024-08-18T13:47:35.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.968+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd80713a0 con 0x7f5fd80c6880 2024-08-18T13:47:35.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.968+0000 7f5fcf7fe700 1 -- 172.21.15.118:0/1357764739 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5fd0003040 con 0x7f5fd80c6880 2024-08-18T13:47:35.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.968+0000 7f5fcf7fe700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5fb8060860 0x7f5fb8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:35.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.968+0000 7f5fcf7fe700 1 -- 172.21.15.118:0/1357764739 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f5fd007ebc0 con 0x7f5fd80c6880 2024-08-18T13:47:35.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.968+0000 7f5fd77fe700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5fb8060860 0x7f5fb8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:35.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.968+0000 7f5fd77fe700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5fb8060860 0x7f5fb8062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009db0 tx=0x7f5fc80091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:35.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:35.976+0000 7f5fcf7fe700 1 -- 172.21.15.118:0/1357764739 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5fd004eb00 con 0x7f5fd80c6880 2024-08-18T13:47:36.202 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.200+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd80c4530 con 0x7f5fd80c6880 2024-08-18T13:47:36.203 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.200+0000 7f5fcf7fe700 1 -- 172.21.15.118:0/1357764739 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f5fd0052120 con 0x7f5fd80c6880 2024-08-18T13:47:36.203 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:36.203 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:33.723760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:36.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5fb8060860 msgr2=0x7f5fb8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:36.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5fb8060860 0x7f5fb8062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009db0 tx=0x7f5fc80091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:36.205 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 msgr2=0x7f5fd814d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd814d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009880 tx=0x7f5fd0018000 comp rx=0 tx=0).stop 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 shutdown_connections 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5fb8060860 0x7f5fb8062d10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5fd80c6880 0x7f5fd814d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fd80aef90 0x7f5fd8157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 --2- 172.21.15.118:0/1357764739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fd80aebc0 0x7f5fd80a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 >> 172.21.15.118:0/1357764739 conn(0x7f5fd80cbba0 msgr2=0x7f5fd80caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 shutdown_connections 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:36.204+0000 7f5fdc8b2700 1 -- 172.21.15.118:0/1357764739 wait complete. 2024-08-18T13:47:36.206 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 28 2024-08-18T13:47:36.217 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:47:38.218 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:38.335 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f718634b700 1 Processor -- start 2024-08-18T13:47:38.335 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f718634b700 1 -- start start 2024-08-18T13:47:38.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f718634b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f71800c3a50 0x7f718014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:38.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f718634b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718006cc90 con 0x7f71800c3e70 2024-08-18T13:47:38.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f718634b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800c83a0 con 0x7f71800c3680 2024-08-18T13:47:38.336 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f718634b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7180067ae0 con 0x7f71800c3a50 2024-08-18T13:47:38.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f7185b4a700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f71800c3e70 0x7f718014d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:52940/0 (socket says 172.21.15.118:52940) 2024-08-18T13:47:38.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f7185b4a700 1 -- 172.21.15.118:0/3410756707 learned_addr learned my addr 172.21.15.118:0/3410756707 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:47:38.337 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f7184b48700 1 --2- 172.21.15.118:0/3410756707 >> v2:172.21.15.178:3300/0 conn(0x7f71800c3a50 0x7f718014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:38.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.332+0000 7f7177fff700 1 -- 172.21.15.118:0/3410756707 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379285855 0 0) 0x7f718006cc90 con 0x7f71800c3e70 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7177fff700 1 -- 172.21.15.118:0/3410756707 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f716c003570 con 0x7f71800c3e70 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7184b48700 1 -- 172.21.15.118:0/3410756707 >> v1:172.21.15.178:6789/0 conn(0x7f71800c3680 legacy=0x7f71800673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7184b48700 1 -- 172.21.15.118:0/3410756707 >> v1:172.21.15.118:6789/0 conn(0x7f71800c3e70 legacy=0x7f718014d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7184b48700 1 -- 172.21.15.118:0/3410756707 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7180153090 con 0x7f71800c3a50 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7184b48700 1 --2- 172.21.15.118:0/3410756707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3a50 0x7f718014cd00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f717c0098e0 tx=0x7f717c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f8926074c1c3d43 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7177fff700 1 -- 172.21.15.118:0/3410756707 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00b640 con 0x7f71800c3a50 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7177fff700 1 -- 172.21.15.118:0/3410756707 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00b7a0 con 0x7f71800c3a50 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f7177fff700 1 -- 172.21.15.118:0/3410756707 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00bb40 con 0x7f71800c3a50 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 -- 172.21.15.118:0/3410756707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3a50 msgr2=0x7f718014cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 --2- 172.21.15.118:0/3410756707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3a50 0x7f718014cd00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f717c0098e0 tx=0x7f717c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 -- 172.21.15.118:0/3410756707 shutdown_connections 2024-08-18T13:47:38.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 --2- 172.21.15.118:0/3410756707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3a50 0x7f718014cd00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 -- 172.21.15.118:0/3410756707 >> 172.21.15.118:0/3410756707 conn(0x7f718006c020 msgr2=0x7f71800c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:38.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 -- 172.21.15.118:0/3410756707 shutdown_connections 2024-08-18T13:47:38.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 -- 172.21.15.118:0/3410756707 wait complete. 2024-08-18T13:47:38.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 Processor -- start 2024-08-18T13:47:38.341 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.336+0000 7f718634b700 1 -- start start 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71800c3680 0x7f71800c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71800c3a50 0x7f71801567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3e70 0x7f7180156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180163c10 con 0x7f71800c3a50 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180158f10 con 0x7f71800c3e70 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180163000 con 0x7f71800c3680 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71800c3a50 0x7f71801567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71800c3a50 0x7f71801567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:49520/0 (socket says 172.21.15.118:49520) 2024-08-18T13:47:38.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 -- 172.21.15.118:0/2646490757 learned_addr learned my addr 172.21.15.118:0/2646490757 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7185349700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71800c3680 0x7f71800c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7185b4a700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3e70 0x7f7180156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 -- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71800c3680 msgr2=0x7f71800c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71800c3680 0x7f71800c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 -- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3e70 msgr2=0x7f7180156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3e70 0x7f7180156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 -- 172.21.15.118:0/2646490757 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c009420 con 0x7f71800c3a50 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f7184b48700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71800c3a50 0x7f71801567a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f718005eb30 tx=0x7f717c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f71757fa700 1 -- 172.21.15.118:0/2646490757 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c000d10 con 0x7f71800c3a50 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f71757fa700 1 -- 172.21.15.118:0/2646490757 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00bb20 con 0x7f71800c3a50 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71801632e0 con 0x7f71800c3a50 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f71757fa700 1 -- 172.21.15.118:0/2646490757 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c011650 con 0x7f71800c3a50 2024-08-18T13:47:38.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.340+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71801638a0 con 0x7f71800c3a50 2024-08-18T13:47:38.346 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.344+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7180163d80 con 0x7f71800c3a50 2024-08-18T13:47:38.346 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.344+0000 7f71757fa700 1 -- 172.21.15.118:0/2646490757 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f717c0119c0 con 0x7f71800c3a50 2024-08-18T13:47:38.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.344+0000 7f71757fa700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7164060860 0x7f7164062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:38.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.344+0000 7f7185349700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7164060860 0x7f7164062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:38.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.344+0000 7f71757fa700 1 -- 172.21.15.118:0/2646490757 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f717c07e460 con 0x7f71800c3a50 2024-08-18T13:47:38.347 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.344+0000 7f7185349700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7164060860 0x7f7164062d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f71700039e0 tx=0x7f7170007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:38.353 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.352+0000 7f71757fa700 1 -- 172.21.15.118:0/2646490757 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f717c04e3a0 con 0x7f71800c3a50 2024-08-18T13:47:38.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.552+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71800713a0 con 0x7f71800c3a50 2024-08-18T13:47:38.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f71757fa700 1 -- 172.21.15.118:0/2646490757 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f717c0519c0 con 0x7f71800c3a50 2024-08-18T13:47:38.558 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:38.558 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:33.723760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:38.560 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7164060860 msgr2=0x7f7164062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7164060860 0x7f7164062d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f71700039e0 tx=0x7f7170007ec0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71800c3a50 msgr2=0x7f71801567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71800c3a50 0x7f71801567a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f718005eb30 tx=0x7f717c017000 comp rx=0 tx=0).stop 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 shutdown_connections 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7164060860 0x7f7164062d10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71800c3e70 0x7f7180156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f71800c3a50 0x7f71801567a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 --2- 172.21.15.118:0/2646490757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71800c3680 0x7f71800c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 >> 172.21.15.118:0/2646490757 conn(0x7f718006c020 msgr2=0x7f718014c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 shutdown_connections 2024-08-18T13:47:38.561 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:38.556+0000 7f718634b700 1 -- 172.21.15.118:0/2646490757 wait complete. 2024-08-18T13:47:38.562 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 28 2024-08-18T13:47:38.572 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:47:40.573 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:40.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7889ac700 1 Processor -- start 2024-08-18T13:47:40.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7889ac700 1 -- start start 2024-08-18T13:47:40.691 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7889ac700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aebc0 0x7fe7840b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:40.692 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7889ac700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe7840c6880 0x7fe7840c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:40.692 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7889ac700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe784155330 con 0x7fe7840aef90 2024-08-18T13:47:40.692 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7889ac700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe784155140 con 0x7fe7840c6880 2024-08-18T13:47:40.692 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7889ac700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840ac1e0 con 0x7fe7840aebc0 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe783fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe7840c6880 0x7fe7840c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe783fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe7840c6880 0x7fe7840c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:49528/0 (socket says 172.21.15.118:49528) 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe783fff700 1 -- 172.21.15.118:0/212437570 learned_addr learned my addr 172.21.15.118:0/212437570 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7837fe700 1 --2- 172.21.15.118:0/212437570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aebc0 0x7fe7840b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe783fff700 1 -- 172.21.15.118:0/212437570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aebc0 msgr2=0x7fe7840b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe7827fc700 1 -- 172.21.15.118:0/212437570 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946978274 0 0) 0x7fe784155330 con 0x7fe7840aef90 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe783fff700 1 --2- 172.21.15.118:0/212437570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aebc0 0x7fe7840b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.688+0000 7fe783fff700 1 -- 172.21.15.118:0/212437570 >> v1:172.21.15.178:6789/0 conn(0x7fe7840aef90 legacy=0x7fe7840abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:40.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe783fff700 1 -- 172.21.15.118:0/212437570 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7841555d0 con 0x7fe7840c6880 2024-08-18T13:47:40.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7837fe700 1 --2- 172.21.15.118:0/212437570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aebc0 0x7fe7840b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:47:40.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe783fff700 1 --2- 172.21.15.118:0/212437570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe7840c6cc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0099b0 tx=0x7fe77c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cae61ec2684c359 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7827fc700 1 -- 172.21.15.118:0/212437570 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c009ea0 con 0x7fe7840c6880 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7827fc700 1 -- 172.21.15.118:0/212437570 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c00b860 con 0x7fe7840c6880 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7827fc700 1 -- 172.21.15.118:0/212437570 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00bc00 con 0x7fe7840c6880 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 -- 172.21.15.118:0/212437570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 msgr2=0x7fe7840c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 --2- 172.21.15.118:0/212437570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe7840c6cc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0099b0 tx=0x7fe77c009250 comp rx=0 tx=0).stop 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 -- 172.21.15.118:0/212437570 shutdown_connections 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 --2- 172.21.15.118:0/212437570 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe7840c6cc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 --2- 172.21.15.118:0/212437570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aebc0 0x7fe7840b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 -- 172.21.15.118:0/212437570 >> 172.21.15.118:0/212437570 conn(0x7fe7840cbba0 msgr2=0x7fe7840cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 -- 172.21.15.118:0/212437570 shutdown_connections 2024-08-18T13:47:40.695 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 -- 172.21.15.118:0/212437570 wait complete. 2024-08-18T13:47:40.696 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 Processor -- start 2024-08-18T13:47:40.696 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.692+0000 7fe7889ac700 1 -- start start 2024-08-18T13:47:40.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7840aebc0 0x7fe784159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:40.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aef90 0x7fe78414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:40.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe78414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:40.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe784153170 con 0x7fe7840c6880 2024-08-18T13:47:40.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840adce0 con 0x7fe7840aebc0 2024-08-18T13:47:40.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78415b690 con 0x7fe7840aef90 2024-08-18T13:47:40.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe78414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:40.699 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe78414aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:49538/0 (socket says 172.21.15.118:49538) 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 -- 172.21.15.118:0/4182665537 learned_addr learned my addr 172.21.15.118:0/4182665537 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe782ffd700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aef90 0x7fe78414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7837fe700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7840aebc0 0x7fe784159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 -- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aef90 msgr2=0x7fe78414d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aef90 0x7fe78414d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 -- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7840aebc0 msgr2=0x7fe784159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7840aebc0 0x7fe784159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 -- 172.21.15.118:0/4182665537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c009420 con 0x7fe7840c6880 2024-08-18T13:47:40.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7837fe700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7840aebc0 0x7fe784159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:47:40.701 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe783fff700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe78414aed0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:40.701 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe77b7fe700 1 -- 172.21.15.118:0/4182665537 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011630 con 0x7fe7840c6880 2024-08-18T13:47:40.701 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe77b7fe700 1 -- 172.21.15.118:0/4182665537 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c0117d0 con 0x7fe7840c6880 2024-08-18T13:47:40.701 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c0094f0 con 0x7fe7840c6880 2024-08-18T13:47:40.701 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe77b7fe700 1 -- 172.21.15.118:0/4182665537 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011b90 con 0x7fe7840c6880 2024-08-18T13:47:40.701 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78415b930 con 0x7fe7840c6880 2024-08-18T13:47:40.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78415bb50 con 0x7fe7840c6880 2024-08-18T13:47:40.703 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.696+0000 7fe77b7fe700 1 -- 172.21.15.118:0/4182665537 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe77c0253f0 con 0x7fe7840c6880 2024-08-18T13:47:40.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.700+0000 7fe77b7fe700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe768060860 0x7fe768062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:40.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.700+0000 7fe77b7fe700 1 -- 172.21.15.118:0/4182665537 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7fe77c07eb80 con 0x7fe7840c6880 2024-08-18T13:47:40.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.700+0000 7fe7837fe700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe768060860 0x7fe768062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:40.704 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.700+0000 7fe7837fe700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe768060860 0x7fe768062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0093f0 tx=0x7fe76c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:40.709 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.708+0000 7fe77b7fe700 1 -- 172.21.15.118:0/4182665537 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe77c04eae0 con 0x7fe7840c6880 2024-08-18T13:47:40.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.908+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7840713a0 con 0x7fe7840c6880 2024-08-18T13:47:40.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.908+0000 7fe77b7fe700 1 -- 172.21.15.118:0/4182665537 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fe77c052100 con 0x7fe7840c6880 2024-08-18T13:47:40.911 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:40.912 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe768060860 msgr2=0x7fe768062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe768060860 0x7fe768062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0093f0 tx=0x7fe76c008040 comp rx=0 tx=0).stop 2024-08-18T13:47:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 msgr2=0x7fe78414aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:40.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe78414aed0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c017000 comp rx=0 tx=0).stop 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 shutdown_connections 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe768060860 0x7fe768062d10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe7840c6880 0x7fe78414aed0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7840aef90 0x7fe78414d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 --2- 172.21.15.118:0/4182665537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7840aebc0 0x7fe784159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 >> 172.21.15.118:0/4182665537 conn(0x7fe7840cbba0 msgr2=0x7fe7840b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 shutdown_connections 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:40.912+0000 7fe7889ac700 1 -- 172.21.15.118:0/4182665537 wait complete. 2024-08-18T13:47:40.915 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 29 2024-08-18T13:47:40.926 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=0, replacing gid=5209 2024-08-18T13:47:40.926 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.f 2024-08-18T13:47:41.927 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-08-18T13:47:41.927 INFO:tasks.ceph.mds.f:Restarting daemon 2024-08-18T13:47:41.928 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-08-18T13:47:41.931 INFO:tasks.ceph.mds.f:Started 2024-08-18T13:47:41.931 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:42.008 INFO:tasks.ceph.mds.f.smithi178.stdout:starting mds.f at 2024-08-18T13:47:42.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 Processor -- start 2024-08-18T13:47:42.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 -- start start 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd9e414f2c0 0x7fd9e406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e405e6b0 0x7fd9e406c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd9e40673d0 0x7fd9e4155150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e40685a0 con 0x7fd9e40673d0 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e40c5910 con 0x7fd9e414f2c0 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e8aa4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4068840 con 0x7fd9e405e6b0 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd9e40673d0 0x7fd9e4155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd9e40673d0 0x7fd9e4155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:49544/0 (socket says 172.21.15.118:49544) 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 -- 172.21.15.118:0/4146190088 learned_addr learned my addr 172.21.15.118:0/4146190088 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e37fe700 1 --2- 172.21.15.118:0/4146190088 >> v2:172.21.15.178:3300/0 conn(0x7fd9e414f2c0 0x7fd9e406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:42.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e2ffd700 1 --2- 172.21.15.118:0/4146190088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e405e6b0 0x7fd9e406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 -- 172.21.15.118:0/4146190088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e405e6b0 msgr2=0x7fd9e406c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 --2- 172.21.15.118:0/4146190088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e405e6b0 0x7fd9e406c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 -- 172.21.15.118:0/4146190088 >> v2:172.21.15.178:3300/0 conn(0x7fd9e414f2c0 msgr2=0x7fd9e406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 --2- 172.21.15.118:0/4146190088 >> v2:172.21.15.178:3300/0 conn(0x7fd9e414f2c0 0x7fd9e406bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 -- 172.21.15.118:0/4146190088 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40c5bf0 con 0x7fd9e40673d0 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e37fe700 1 --2- 172.21.15.118:0/4146190088 >> v2:172.21.15.178:3300/0 conn(0x7fd9e414f2c0 0x7fd9e406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e2ffd700 1 --2- 172.21.15.118:0/4146190088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e405e6b0 0x7fd9e406c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e3fff700 1 --2- 172.21.15.118:0/4146190088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e40673d0 0x7fd9e4155150 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd9e406d8a0 tx=0x7fd9dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c6dc210c2131aa server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e27fc700 1 -- 172.21.15.118:0/4146190088 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00b860 con 0x7fd9e40673d0 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e27fc700 1 -- 172.21.15.118:0/4146190088 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc00b9c0 con 0x7fd9e40673d0 2024-08-18T13:47:42.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.048+0000 7fd9e27fc700 1 -- 172.21.15.118:0/4146190088 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00bd60 con 0x7fd9e40673d0 2024-08-18T13:47:42.055 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/4146190088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e40673d0 msgr2=0x7fd9e4155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:42.055 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/4146190088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e40673d0 0x7fd9e4155150 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd9e406d8a0 tx=0x7fd9dc009250 comp rx=0 tx=0).stop 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/4146190088 shutdown_connections 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/4146190088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e40673d0 0x7fd9e4155150 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/4146190088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e405e6b0 0x7fd9e406c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/4146190088 >> v2:172.21.15.178:3300/0 conn(0x7fd9e414f2c0 0x7fd9e406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/4146190088 >> 172.21.15.118:0/4146190088 conn(0x7fd9e406af80 msgr2=0x7fd9e406e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/4146190088 shutdown_connections 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/4146190088 wait complete. 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 Processor -- start 2024-08-18T13:47:42.056 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 -- start start 2024-08-18T13:47:42.058 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.052+0000 7fd9e8aa4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e405e6b0 0x7fd9e40c1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9e40673d0 0x7fd9e4158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e414f2c0 0x7fd9e4145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e41663e0 con 0x7fd9e405e6b0 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4165580 con 0x7fd9e40673d0 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4165860 con 0x7fd9e414f2c0 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e405e6b0 0x7fd9e40c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e405e6b0 0x7fd9e40c1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:49558/0 (socket says 172.21.15.118:49558) 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 -- 172.21.15.118:0/956066160 learned_addr learned my addr 172.21.15.118:0/956066160 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e3fff700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e414f2c0 0x7fd9e4145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e2ffd700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9e40673d0 0x7fd9e4158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 -- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e414f2c0 msgr2=0x7fd9e4145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:42.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e414f2c0 0x7fd9e4145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 -- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9e40673d0 msgr2=0x7fd9e4158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9e40673d0 0x7fd9e4158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 -- 172.21.15.118:0/956066160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0094f0 con 0x7fd9e405e6b0 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e3fff700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e414f2c0 0x7fd9e4145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e37fe700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e405e6b0 0x7fd9e40c1800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d860 tx=0x7fd9d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9db7fe700 1 -- 172.21.15.118:0/956066160 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4009610 con 0x7fd9e405e6b0 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9db7fe700 1 -- 172.21.15.118:0/956066160 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4009770 con 0x7fd9e405e6b0 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e4165b40 con 0x7fd9e405e6b0 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9db7fe700 1 -- 172.21.15.118:0/956066160 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4009b10 con 0x7fd9e405e6b0 2024-08-18T13:47:42.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e4166100 con 0x7fd9e405e6b0 2024-08-18T13:47:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e40713a0 con 0x7fd9e405e6b0 2024-08-18T13:47:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.056+0000 7fd9db7fe700 1 -- 172.21.15.118:0/956066160 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd9d4014430 con 0x7fd9e405e6b0 2024-08-18T13:47:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.060+0000 7fd9db7fe700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd9c4060860 0x7fd9c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.060+0000 7fd9e2ffd700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd9c4060860 0x7fd9c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:42.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.060+0000 7fd9db7fe700 1 -- 172.21.15.118:0/956066160 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7fd9d407d5e0 con 0x7fd9e405e6b0 2024-08-18T13:47:42.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.060+0000 7fd9e2ffd700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd9c4060860 0x7fd9c4062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4151af0 tx=0x7fd9cc0098f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:42.065 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.064+0000 7fd9db7fe700 1 -- 172.21.15.118:0/956066160 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd9d404d480 con 0x7fd9e405e6b0 2024-08-18T13:47:42.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.264+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9e40bf7c0 con 0x7fd9e405e6b0 2024-08-18T13:47:42.266 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.264+0000 7fd9db7fe700 1 -- 172.21.15.118:0/956066160 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fd9d4050aa0 con 0x7fd9e405e6b0 2024-08-18T13:47:42.267 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:42.267 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:42.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd9c4060860 msgr2=0x7fd9c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:42.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd9c4060860 0x7fd9c4062d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4151af0 tx=0x7fd9cc0098f0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e405e6b0 msgr2=0x7fd9e40c1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e405e6b0 0x7fd9e40c1800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d860 tx=0x7fd9d4008040 comp rx=0 tx=0).stop 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 shutdown_connections 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd9c4060860 0x7fd9c4062d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9e414f2c0 0x7fd9e4145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9e40673d0 0x7fd9e4158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 --2- 172.21.15.118:0/956066160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9e405e6b0 0x7fd9e40c1800 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 >> 172.21.15.118:0/956066160 conn(0x7fd9e406af80 msgr2=0x7fd9e40c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 shutdown_connections 2024-08-18T13:47:42.270 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:42.268+0000 7fd9e8aa4700 1 -- 172.21.15.118:0/956066160 wait complete. 2024-08-18T13:47:42.271 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 29 2024-08-18T13:47:42.281 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-08-18T13:47:43.340 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:47:43.343 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:47:44.281 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:47:44.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f959be5a700 1 Processor -- start 2024-08-18T13:47:44.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f959be5a700 1 -- start start 2024-08-18T13:47:44.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f959be5a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f95940be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:44.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f959be5a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f959415a310 0x7f959406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:44.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f959be5a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95940c3020 con 0x7f95940c3d50 2024-08-18T13:47:44.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f959be5a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95940c38c0 con 0x7f959415a310 2024-08-18T13:47:44.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f959be5a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959406e3b0 con 0x7f95940be340 2024-08-18T13:47:44.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f95993f5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f959415a310 0x7f959406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:44.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f9599bf6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f95940be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:44.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f95993f5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f959415a310 0x7f959406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:49084/0 (socket says 172.21.15.118:49084) 2024-08-18T13:47:44.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f95993f5700 1 -- 172.21.15.118:0/3500768519 learned_addr learned my addr 172.21.15.118:0/3500768519 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:44.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f95993f5700 1 -- 172.21.15.118:0/3500768519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 msgr2=0x7f95940be710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:47:44.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f95993f5700 1 --2- 172.21.15.118:0/3500768519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f95940be710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f95993f5700 1 -- 172.21.15.118:0/3500768519 >> v1:172.21.15.178:6789/0 conn(0x7f95940c3d50 legacy=0x7f959406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:44.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.392+0000 7f95993f5700 1 -- 172.21.15.118:0/3500768519 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95940673d0 con 0x7f959415a310 2024-08-18T13:47:44.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f95993f5700 1 --2- 172.21.15.118:0/3500768519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959415a310 0x7f959406db20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9590009f20 tx=0x7f9590009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2297035549e3ce server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:44.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f9598bf4700 1 -- 172.21.15.118:0/3500768519 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959000e070 con 0x7f959415a310 2024-08-18T13:47:44.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f9598bf4700 1 -- 172.21.15.118:0/3500768519 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959000b9d0 con 0x7f959415a310 2024-08-18T13:47:44.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f9598bf4700 1 -- 172.21.15.118:0/3500768519 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95900092e0 con 0x7f959415a310 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 -- 172.21.15.118:0/3500768519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959415a310 msgr2=0x7f959406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 --2- 172.21.15.118:0/3500768519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959415a310 0x7f959406db20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9590009f20 tx=0x7f9590009840 comp rx=0 tx=0).stop 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 -- 172.21.15.118:0/3500768519 shutdown_connections 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 --2- 172.21.15.118:0/3500768519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959415a310 0x7f959406db20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 --2- 172.21.15.118:0/3500768519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f95940be710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 -- 172.21.15.118:0/3500768519 >> 172.21.15.118:0/3500768519 conn(0x7f95940c7c40 msgr2=0x7f95940c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 -- 172.21.15.118:0/3500768519 shutdown_connections 2024-08-18T13:47:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.396+0000 7f959be5a700 1 -- 172.21.15.118:0/3500768519 wait complete. 2024-08-18T13:47:44.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 Processor -- start 2024-08-18T13:47:44.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 -- start start 2024-08-18T13:47:44.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f959416af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:44.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940c3d50 0x7f959415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:44.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f959415a310 0x7f959416d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:44.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959416aa10 con 0x7f959415a310 2024-08-18T13:47:44.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959416a4e0 con 0x7f95940c3d50 2024-08-18T13:47:44.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9594154c50 con 0x7f95940be340 2024-08-18T13:47:44.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f959415a310 0x7f959416d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:44.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f959415a310 0x7f959416d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53032/0 (socket says 172.21.15.118:53032) 2024-08-18T13:47:44.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 -- 172.21.15.118:0/944864984 learned_addr learned my addr 172.21.15.118:0/944864984 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:47:44.406 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f95993f5700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940c3d50 0x7f959415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f9599bf6700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f959416af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 -- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 msgr2=0x7f959416af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f959416af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 -- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940c3d50 msgr2=0x7f959415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940c3d50 0x7f959415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 -- 172.21.15.118:0/944864984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9590009ae0 con 0x7f959415a310 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959a3f7700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f959415a310 0x7f959416d060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c590 tx=0x7f958c007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:44.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f9589ffb700 1 -- 172.21.15.118:0/944864984 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c011590 con 0x7f959415a310 2024-08-18T13:47:44.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f9589ffb700 1 -- 172.21.15.118:0/944864984 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c011710 con 0x7f959415a310 2024-08-18T13:47:44.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9594154360 con 0x7f959415a310 2024-08-18T13:47:44.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f9589ffb700 1 -- 172.21.15.118:0/944864984 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c011ad0 con 0x7f959415a310 2024-08-18T13:47:44.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959416a730 con 0x7f959415a310 2024-08-18T13:47:44.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.400+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9594154de0 con 0x7f959415a310 2024-08-18T13:47:44.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.404+0000 7f9589ffb700 1 -- 172.21.15.118:0/944864984 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f958c01b3b0 con 0x7f959415a310 2024-08-18T13:47:44.410 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.404+0000 7f9589ffb700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9580060860 0x7f9580062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:47:44.410 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.404+0000 7f9589ffb700 1 -- 172.21.15.118:0/944864984 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f958c07c890 con 0x7f959415a310 2024-08-18T13:47:44.410 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.404+0000 7f9599bf6700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9580060860 0x7f9580062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:47:44.410 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.404+0000 7f9599bf6700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9580060860 0x7f9580062d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9584009d40 tx=0x7f9584008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:47:44.415 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.412+0000 7f9589ffb700 1 -- 172.21.15.118:0/944864984 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f958c04c850 con 0x7f959415a310 2024-08-18T13:47:44.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.612+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95940642f0 con 0x7f959415a310 2024-08-18T13:47:44.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f9589ffb700 1 -- 172.21.15.118:0/944864984 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f958c017020 con 0x7f959415a310 2024-08-18T13:47:44.618 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:47:44.618 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:47:44.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9580060860 msgr2=0x7f9580062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:44.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9580060860 0x7f9580062d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9584009d40 tx=0x7f9584008040 comp rx=0 tx=0).stop 2024-08-18T13:47:44.621 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f959415a310 msgr2=0x7f959416d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:47:44.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f959415a310 0x7f959416d060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c590 tx=0x7f958c007eb0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 shutdown_connections 2024-08-18T13:47:44.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9580060860 0x7f9580062d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f959415a310 0x7f959416d060 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940c3d50 0x7f959415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 --2- 172.21.15.118:0/944864984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940be340 0x7f959416af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:47:44.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 >> 172.21.15.118:0/944864984 conn(0x7f95940c7c40 msgr2=0x7f9594067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:47:44.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 shutdown_connections 2024-08-18T13:47:44.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:47:44.616+0000 7f959be5a700 1 -- 172.21.15.118:0/944864984 wait complete. 2024-08-18T13:47:44.623 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:47:44.632 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-08-18T13:47:44.632 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 89 secs before thrashing 2024-08-18T13:47:49.503 INFO:tasks.ceph.mds.d.smithi178.stderr:2024-08-18T13:47:49.497+0000 7fdd5c8a7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T13:48:13.622 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:48:13.625 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:48:43.675 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:48:43.678 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:49:13.633 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:13.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 Processor -- start 2024-08-18T13:49:13.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- start start 2024-08-18T13:49:13.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5b700aebc0 0x7f5b700b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:13.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700c6880 0x7f5b700c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b70155330 con 0x7f5b700aef90 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b70155140 con 0x7f5b700aebc0 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b700ac1e0 con 0x7f5b700c6880 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5b700aebc0 0x7f5b700b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f5b700aebc0 0x7f5b700b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53092/0 (socket says 172.21.15.118:53092) 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 -- 172.21.15.118:0/1113322066 learned_addr learned my addr 172.21.15.118:0/1113322066 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b75a4e700 1 --2- 172.21.15.118:0/1113322066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700c6880 0x7f5b700c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 -- 172.21.15.118:0/1113322066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700c6880 msgr2=0x7f5b700c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 --2- 172.21.15.118:0/1113322066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700c6880 0x7f5b700c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 -- 172.21.15.118:0/1113322066 >> v1:172.21.15.178:6789/0 conn(0x7f5b700aef90 legacy=0x7f5b700abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 -- 172.21.15.118:0/1113322066 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b701555d0 con 0x7f5b700aebc0 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7524d700 1 --2- 172.21.15.118:0/1113322066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aebc0 0x7f5b700b1640 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b640099e0 tx=0x7f5b64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f334c98a53bd5c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b6ffff700 1 -- 172.21.15.118:0/1113322066 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400b820 con 0x7f5b700aebc0 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b6ffff700 1 -- 172.21.15.118:0/1113322066 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6400b980 con 0x7f5b700aebc0 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b6ffff700 1 -- 172.21.15.118:0/1113322066 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400bd20 con 0x7f5b700aebc0 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- 172.21.15.118:0/1113322066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aebc0 msgr2=0x7f5b700b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 --2- 172.21.15.118:0/1113322066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aebc0 0x7f5b700b1640 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b640099e0 tx=0x7f5b64009250 comp rx=0 tx=0).stop 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- 172.21.15.118:0/1113322066 shutdown_connections 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 --2- 172.21.15.118:0/1113322066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700c6880 0x7f5b700c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 --2- 172.21.15.118:0/1113322066 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aebc0 0x7f5b700b1640 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- 172.21.15.118:0/1113322066 >> 172.21.15.118:0/1113322066 conn(0x7f5b700cbba0 msgr2=0x7f5b700cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:13.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- 172.21.15.118:0/1113322066 shutdown_connections 2024-08-18T13:49:13.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.746+0000 7f5b7624f700 1 -- 172.21.15.118:0/1113322066 wait complete. 2024-08-18T13:49:13.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 Processor -- start 2024-08-18T13:49:13.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 -- start start 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700aebc0 0x7f5b700ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aef90 0x7f5b701687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b700c6880 0x7f5b700c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b701696b0 con 0x7f5b700aef90 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b700c22b0 con 0x7f5b700c6880 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70168e10 con 0x7f5b700aebc0 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aef90 0x7f5b701687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aef90 0x7f5b701687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:53108/0 (socket says 172.21.15.118:53108) 2024-08-18T13:49:13.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7524d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700aebc0 0x7f5b700ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:13.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 -- 172.21.15.118:0/2308784375 learned_addr learned my addr 172.21.15.118:0/2308784375 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:13.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b75a4e700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b700c6880 0x7f5b700c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:13.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 -- 172.21.15.118:0/2308784375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700aebc0 msgr2=0x7f5b700ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:13.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700aebc0 0x7f5b700ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.759 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 -- 172.21.15.118:0/2308784375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b700c6880 msgr2=0x7f5b700c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b700c6880 0x7f5b700c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 -- 172.21.15.118:0/2308784375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b640094f0 con 0x7f5b700aef90 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b74a4c700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aef90 0x7f5b701687a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800cb10 tx=0x7f5b680017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b6d7fa700 1 -- 172.21.15.118:0/2308784375 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800ce20 con 0x7f5b700aef90 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b6d7fa700 1 -- 172.21.15.118:0/2308784375 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68006ba0 con 0x7f5b700aef90 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b701690f0 con 0x7f5b700aef90 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b6d7fa700 1 -- 172.21.15.118:0/2308784375 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800a7e0 con 0x7f5b700aef90 2024-08-18T13:49:13.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.750+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b700c2f90 con 0x7f5b700aef90 2024-08-18T13:49:13.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.754+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b500052b0 con 0x7f5b700aef90 2024-08-18T13:49:13.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.754+0000 7f5b6d7fa700 1 -- 172.21.15.118:0/2308784375 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5b6800abd0 con 0x7f5b700aef90 2024-08-18T13:49:13.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.754+0000 7f5b6d7fa700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5b5c060860 0x7f5b5c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:13.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.754+0000 7f5b7524d700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5b5c060860 0x7f5b5c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:13.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.754+0000 7f5b6d7fa700 1 -- 172.21.15.118:0/2308784375 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f5b6807cde0 con 0x7f5b700aef90 2024-08-18T13:49:13.762 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.754+0000 7f5b7524d700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5b5c060860 0x7f5b5c062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b640099e0 tx=0x7f5b64017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:13.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.762+0000 7f5b6d7fa700 1 -- 172.21.15.118:0/2308784375 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5b6804cd20 con 0x7f5b700aef90 2024-08-18T13:49:13.982 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.978+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b50005560 con 0x7f5b700aef90 2024-08-18T13:49:13.983 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.978+0000 7f5b6d7fa700 1 -- 172.21.15.118:0/2308784375 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f5b68016090 con 0x7f5b700aef90 2024-08-18T13:49:13.984 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:13.984 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:13.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5b5c060860 msgr2=0x7f5b5c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:13.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5b5c060860 0x7f5b5c062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b640099e0 tx=0x7f5b64017000 comp rx=0 tx=0).stop 2024-08-18T13:49:13.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aef90 msgr2=0x7f5b701687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:13.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aef90 0x7f5b701687a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800cb10 tx=0x7f5b680017b0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 shutdown_connections 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5b5c060860 0x7f5b5c062d10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b700c6880 0x7f5b700c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b700aef90 0x7f5b701687a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 --2- 172.21.15.118:0/2308784375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b700aebc0 0x7f5b700ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 >> 172.21.15.118:0/2308784375 conn(0x7f5b700cbba0 msgr2=0x7f5b7014f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 shutdown_connections 2024-08-18T13:49:13.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:13.982+0000 7f5b7624f700 1 -- 172.21.15.118:0/2308784375 wait complete. 2024-08-18T13:49:13.988 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:14.003 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=0) 2024-08-18T13:49:14.003 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-18T13:49:14.003 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:49:14.020 INFO:tasks.ceph.mds.d:Stopped 2024-08-18T13:49:14.020 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:14.056 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:49:14.059 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:49:14.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72f4cde700 1 Processor -- start 2024-08-18T13:49:14.126 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72f4cde700 1 -- start start 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72f4cde700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f72f00c3680 0x7f72f00673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72f4cde700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f72f00c3a50 0x7f72f014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72f4cde700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f006cc90 con 0x7f72f00c3e70 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72f4cde700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f00c83a0 con 0x7f72f00c3680 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72f4cde700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f0153090 con 0x7f72f00c3a50 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72effff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f72f00c3e70 0x7f72f014d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41714/0 (socket says 172.21.15.118:41714) 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72effff700 1 -- 172.21.15.118:0/2147982630 learned_addr learned my addr 172.21.15.118:0/2147982630 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:14.129 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ef7fe700 1 --2- 172.21.15.118:0/2147982630 >> v2:172.21.15.118:3300/0 conn(0x7f72f00c3680 0x7f72f00673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72eeffd700 1 --2- 172.21.15.118:0/2147982630 >> v2:172.21.15.178:3300/0 conn(0x7f72f00c3a50 0x7f72f014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ee7fc700 1 -- 172.21.15.118:0/2147982630 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173314396 0 0) 0x7f72f006cc90 con 0x7f72f00c3e70 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ee7fc700 1 -- 172.21.15.118:0/2147982630 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d0003570 con 0x7f72f00c3e70 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ef7fe700 1 -- 172.21.15.118:0/2147982630 >> v2:172.21.15.178:3300/0 conn(0x7f72f00c3a50 msgr2=0x7f72f014cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ee7fc700 1 -- 172.21.15.118:0/2147982630 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3449336444 0 0) 0x7f72d0003570 con 0x7f72f00c3e70 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ef7fe700 1 --2- 172.21.15.118:0/2147982630 >> v2:172.21.15.178:3300/0 conn(0x7f72f00c3a50 0x7f72f014cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ef7fe700 1 -- 172.21.15.118:0/2147982630 >> v1:172.21.15.118:6789/0 conn(0x7f72f00c3e70 legacy=0x7f72f014d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ef7fe700 1 -- 172.21.15.118:0/2147982630 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f00c8660 con 0x7f72f00c3680 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.118+0000 7f72ef7fe700 1 --2- 172.21.15.118:0/2147982630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3680 0x7f72f00673d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f72f0068e50 tx=0x7f72e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdfb10a9044ded server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:14.130 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72ee7fc700 1 -- 172.21.15.118:0/2147982630 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8001470 con 0x7f72f00c3680 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72ee7fc700 1 -- 172.21.15.118:0/2147982630 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0011420 con 0x7f72f00c3680 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72ee7fc700 1 -- 172.21.15.118:0/2147982630 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e00117c0 con 0x7f72f00c3680 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- 172.21.15.118:0/2147982630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3680 msgr2=0x7f72f00673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 --2- 172.21.15.118:0/2147982630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3680 0x7f72f00673d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f72f0068e50 tx=0x7f72e00091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- 172.21.15.118:0/2147982630 shutdown_connections 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 --2- 172.21.15.118:0/2147982630 >> v2:172.21.15.178:3300/0 conn(0x7f72f00c3a50 0x7f72f014cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 --2- 172.21.15.118:0/2147982630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3680 0x7f72f00673d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.131 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- 172.21.15.118:0/2147982630 >> 172.21.15.118:0/2147982630 conn(0x7f72f006c020 msgr2=0x7f72f00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:14.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- 172.21.15.118:0/2147982630 shutdown_connections 2024-08-18T13:49:14.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- 172.21.15.118:0/2147982630 wait complete. 2024-08-18T13:49:14.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 Processor -- start 2024-08-18T13:49:14.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- start start 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72f00c3680 0x7f72f014cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3a50 0x7f72f014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72f00c3e70 0x7f72f014f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f014f370 con 0x7f72f00c3a50 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0165f30 con 0x7f72f00c3e70 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72f4cde700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f014ea40 con 0x7f72f00c3680 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72eeffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3a50 0x7f72f014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:14.133 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72eeffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3a50 0x7f72f014d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45088/0 (socket says 172.21.15.118:45088) 2024-08-18T13:49:14.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72eeffd700 1 -- 172.21.15.118:0/4125847783 learned_addr learned my addr 172.21.15.118:0/4125847783 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:14.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72effff700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72f00c3e70 0x7f72f014f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:14.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.122+0000 7f72ef7fe700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72f00c3680 0x7f72f014cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:14.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72eeffd700 1 -- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72f00c3680 msgr2=0x7f72f014cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:14.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72eeffd700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72f00c3680 0x7f72f014cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.134 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72eeffd700 1 -- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72f00c3e70 msgr2=0x7f72f014f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:14.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72eeffd700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72f00c3e70 0x7f72f014f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72eeffd700 1 -- 172.21.15.118:0/4125847783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0009420 con 0x7f72f00c3a50 2024-08-18T13:49:14.135 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72effff700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72f00c3e70 0x7f72f014f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:49:14.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72eeffd700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3a50 0x7f72f014d1c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009e50 tx=0x7f72d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:14.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72e77fe700 1 -- 172.21.15.118:0/4125847783 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ccb0 con 0x7f72f00c3a50 2024-08-18T13:49:14.137 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72e77fe700 1 -- 172.21.15.118:0/4125847783 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d800f040 con 0x7f72f00c3a50 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f014ed20 con 0x7f72f00c3a50 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72e77fe700 1 -- 172.21.15.118:0/4125847783 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d80146b0 con 0x7f72f00c3a50 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f014e740 con 0x7f72f00c3a50 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f00713a0 con 0x7f72f00c3a50 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.126+0000 7f72e77fe700 1 -- 172.21.15.118:0/4125847783 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f72d8014a20 con 0x7f72f00c3a50 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.130+0000 7f72e77fe700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f72d4060860 0x7f72d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.130+0000 7f72ef7fe700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f72d4060860 0x7f72d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.130+0000 7f72e77fe700 1 -- 172.21.15.118:0/4125847783 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f72d807de40 con 0x7f72f00c3a50 2024-08-18T13:49:14.138 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.130+0000 7f72ef7fe700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f72d4060860 0x7f72d4062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72f0068e50 tx=0x7f72e0016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:14.139 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.134+0000 7f72e77fe700 1 -- 172.21.15.118:0/4125847783 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f72d804dd80 con 0x7f72f00c3a50 2024-08-18T13:49:14.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.334+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72f00675e0 con 0x7f72f00c3a50 2024-08-18T13:49:14.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.334+0000 7f72e77fe700 1 -- 172.21.15.118:0/4125847783 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f72d80513a0 con 0x7f72f00c3a50 2024-08-18T13:49:14.340 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:14.341 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:14.343 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f72d4060860 msgr2=0x7f72d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:14.343 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f72d4060860 0x7f72d4062d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72f0068e50 tx=0x7f72e0016000 comp rx=0 tx=0).stop 2024-08-18T13:49:14.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3a50 msgr2=0x7f72f014d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:14.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3a50 0x7f72f014d1c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009e50 tx=0x7f72d800b040 comp rx=0 tx=0).stop 2024-08-18T13:49:14.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 shutdown_connections 2024-08-18T13:49:14.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f72d4060860 0x7f72d4062d10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72f00c3e70 0x7f72f014f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f72f00c3a50 0x7f72f014d1c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 --2- 172.21.15.118:0/4125847783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72f00c3680 0x7f72f014cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:14.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 >> 172.21.15.118:0/4125847783 conn(0x7f72f006c020 msgr2=0x7f72f014b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:14.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 shutdown_connections 2024-08-18T13:49:14.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:14.338+0000 7f72f4cde700 1 -- 172.21.15.118:0/4125847783 wait complete. 2024-08-18T13:49:14.345 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:14.354 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:49:16.355 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:16.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37bc22b700 1 Processor -- start 2024-08-18T13:49:16.474 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37bc22b700 1 -- start start 2024-08-18T13:49:16.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37bc22b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f37b40aee00 0x7f37b415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:16.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37bc22b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b4153d10 0x7f37b40c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:16.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37bc22b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b415a720 con 0x7f37b40acf70 2024-08-18T13:49:16.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37bc22b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b40ad700 con 0x7f37b40aee00 2024-08-18T13:49:16.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37bc22b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40c6f90 con 0x7f37b4153d10 2024-08-18T13:49:16.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b97c6700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f37b40acf70 0x7f37b4153600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41728/0 (socket says 172.21.15.118:41728) 2024-08-18T13:49:16.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b97c6700 1 -- 172.21.15.118:0/2354095166 learned_addr learned my addr 172.21.15.118:0/2354095166 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:16.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b9fc7700 1 --2- 172.21.15.118:0/2354095166 >> v2:172.21.15.118:3300/0 conn(0x7f37b40aee00 0x7f37b415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:16.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37ba7c8700 1 --2- 172.21.15.118:0/2354095166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b4153d10 0x7f37b40c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b8fc5700 1 -- 172.21.15.118:0/2354095166 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664776979 0 0) 0x7f37b415a720 con 0x7f37b40acf70 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b8fc5700 1 -- 172.21.15.118:0/2354095166 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a0003570 con 0x7f37b40acf70 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b9fc7700 1 -- 172.21.15.118:0/2354095166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b4153d10 msgr2=0x7f37b40c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b9fc7700 1 --2- 172.21.15.118:0/2354095166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b4153d10 0x7f37b40c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b9fc7700 1 -- 172.21.15.118:0/2354095166 >> v1:172.21.15.118:6789/0 conn(0x7f37b40acf70 legacy=0x7f37b4153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b9fc7700 1 -- 172.21.15.118:0/2354095166 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40c7760 con 0x7f37b40aee00 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b9fc7700 1 --2- 172.21.15.118:0/2354095166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415a1e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37b00098e0 tx=0x7f37b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8423a0c46d9317bd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b8fc5700 1 -- 172.21.15.118:0/2354095166 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0011630 con 0x7f37b40aee00 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b8fc5700 1 -- 172.21.15.118:0/2354095166 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b0011790 con 0x7f37b40aee00 2024-08-18T13:49:16.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.470+0000 7f37b8fc5700 1 -- 172.21.15.118:0/2354095166 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0011b30 con 0x7f37b40aee00 2024-08-18T13:49:16.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 -- 172.21.15.118:0/2354095166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 msgr2=0x7f37b415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:16.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 --2- 172.21.15.118:0/2354095166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415a1e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37b00098e0 tx=0x7f37b00091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 -- 172.21.15.118:0/2354095166 shutdown_connections 2024-08-18T13:49:16.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 --2- 172.21.15.118:0/2354095166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b4153d10 0x7f37b40c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 --2- 172.21.15.118:0/2354095166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415a1e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 -- 172.21.15.118:0/2354095166 >> 172.21.15.118:0/2354095166 conn(0x7f37b40ca7b0 msgr2=0x7f37b40c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:16.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 -- 172.21.15.118:0/2354095166 shutdown_connections 2024-08-18T13:49:16.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 -- 172.21.15.118:0/2354095166 wait complete. 2024-08-18T13:49:16.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 Processor -- start 2024-08-18T13:49:16.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.474+0000 7f37bc22b700 1 -- start start 2024-08-18T13:49:16.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b40acf70 0x7f37b4154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:16.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:16.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b4153d10 0x7f37b415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:16.483 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4156cc0 con 0x7f37b40aee00 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41570e0 con 0x7f37b4153d10 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4154730 con 0x7f37b40acf70 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45104/0 (socket says 172.21.15.118:45104) 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 -- 172.21.15.118:0/105485250 learned_addr learned my addr 172.21.15.118:0/105485250 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b9fc7700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b40acf70 0x7f37b4154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37ba7c8700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b4153d10 0x7f37b415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 -- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b40acf70 msgr2=0x7f37b4154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b40acf70 0x7f37b4154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 -- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b4153d10 msgr2=0x7f37b415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b4153d10 0x7f37b415b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 -- 172.21.15.118:0/105485250 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b0009420 con 0x7f37b40aee00 2024-08-18T13:49:16.484 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b9fc7700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b40acf70 0x7f37b4154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37ba7c8700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b4153d10 0x7f37b415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:49:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37b97c6700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415ab60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00cd90 tx=0x7f37ac001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37a5ffb700 1 -- 172.21.15.118:0/105485250 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00ab80 con 0x7f37b40aee00 2024-08-18T13:49:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37a5ffb700 1 -- 172.21.15.118:0/105485250 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00ad00 con 0x7f37b40aee00 2024-08-18T13:49:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b40ac810 con 0x7f37b40aee00 2024-08-18T13:49:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37a5ffb700 1 -- 172.21.15.118:0/105485250 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac007340 con 0x7f37b40aee00 2024-08-18T13:49:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b416a3f0 con 0x7f37b40aee00 2024-08-18T13:49:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.478+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b40648f0 con 0x7f37b40aee00 2024-08-18T13:49:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.482+0000 7f37a5ffb700 1 -- 172.21.15.118:0/105485250 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f37ac007730 con 0x7f37b40aee00 2024-08-18T13:49:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.482+0000 7f37a5ffb700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3798060860 0x7f3798062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.482+0000 7f37a5ffb700 1 -- 172.21.15.118:0/105485250 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f37ac07d680 con 0x7f37b40aee00 2024-08-18T13:49:16.486 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.482+0000 7f37b9fc7700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3798060860 0x7f3798062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:16.487 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.482+0000 7f37b9fc7700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3798060860 0x7f3798062d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37b00091e0 tx=0x7f37b0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:16.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.486+0000 7f37a5ffb700 1 -- 172.21.15.118:0/105485250 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f37ac04d640 con 0x7f37b40aee00 2024-08-18T13:49:16.693 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37b415a660 con 0x7f37b40aee00 2024-08-18T13:49:16.694 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37a5ffb700 1 -- 172.21.15.118:0/105485250 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f37ac050c60 con 0x7f37b40aee00 2024-08-18T13:49:16.694 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3798060860 msgr2=0x7f3798062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3798060860 0x7f3798062d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37b00091e0 tx=0x7f37b0017000 comp rx=0 tx=0).stop 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 msgr2=0x7f37b415ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415ab60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00cd90 tx=0x7f37ac001b10 comp rx=0 tx=0).stop 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 shutdown_connections 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3798060860 0x7f3798062d10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b4153d10 0x7f37b415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f37b40aee00 0x7f37b415ab60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.697 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 --2- 172.21.15.118:0/105485250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37b40acf70 0x7f37b4154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:16.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 >> 172.21.15.118:0/105485250 conn(0x7f37b40ca7b0 msgr2=0x7f37b406d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:16.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 shutdown_connections 2024-08-18T13:49:16.698 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:16.690+0000 7f37bc22b700 1 -- 172.21.15.118:0/105485250 wait complete. 2024-08-18T13:49:16.698 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:16.708 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:49:18.709 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:18.824 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.818+0000 7fe16f59e700 1 Processor -- start 2024-08-18T13:49:18.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.818+0000 7fe16f59e700 1 -- start start 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3a50 0x7fe17014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1700c3e40 0x7fe17014d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17006cc90 con 0x7fe1700c3680 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe170067ae0 con 0x7fe1700c3e40 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17014d780 con 0x7fe1700c3a50 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16dd9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3a50 0x7fe17014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1700c3e40 0x7fe17014d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1700c3e40 0x7fe17014d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:32810/0 (socket says 172.21.15.118:32810) 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 -- 172.21.15.118:0/2226433126 learned_addr learned my addr 172.21.15.118:0/2226433126 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:18.826 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16e59c700 1 --1- 172.21.15.118:0/2226433126 >> v1:172.21.15.178:6789/0 conn(0x7fe1700c3680 0x7fe1700673d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:57106/0 (socket says 172.21.15.118:57106) 2024-08-18T13:49:18.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16dd9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3a50 0x7fe17014cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:52070/0 (socket says 172.21.15.118:52070) 2024-08-18T13:49:18.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 -- 172.21.15.118:0/2226433126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3a50 msgr2=0x7fe17014cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:18.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 --2- 172.21.15.118:0/2226433126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3a50 0x7fe17014cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:18.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 -- 172.21.15.118:0/2226433126 >> v1:172.21.15.178:6789/0 conn(0x7fe1700c3680 legacy=0x7fe1700673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:18.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 -- 172.21.15.118:0/2226433126 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700c83a0 con 0x7fe1700c3e40 2024-08-18T13:49:18.828 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16ed9d700 1 --2- 172.21.15.118:0/2226433126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3e40 0x7fe17014d240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1680099b0 tx=0x7fe168009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a43a9d2b777732e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16d59a700 1 -- 172.21.15.118:0/2226433126 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168009ea0 con 0x7fe1700c3e40 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16d59a700 1 -- 172.21.15.118:0/2226433126 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16800b860 con 0x7fe1700c3e40 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16d59a700 1 -- 172.21.15.118:0/2226433126 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800bc00 con 0x7fe1700c3e40 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- 172.21.15.118:0/2226433126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3e40 msgr2=0x7fe17014d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 --2- 172.21.15.118:0/2226433126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3e40 0x7fe17014d240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1680099b0 tx=0x7fe168009250 comp rx=0 tx=0).stop 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- 172.21.15.118:0/2226433126 shutdown_connections 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 --2- 172.21.15.118:0/2226433126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3e40 0x7fe17014d240 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 --2- 172.21.15.118:0/2226433126 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3a50 0x7fe17014cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- 172.21.15.118:0/2226433126 >> 172.21.15.118:0/2226433126 conn(0x7fe17006c020 msgr2=0x7fe1700c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- 172.21.15.118:0/2226433126 shutdown_connections 2024-08-18T13:49:18.829 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.822+0000 7fe16f59e700 1 -- 172.21.15.118:0/2226433126 wait complete. 2024-08-18T13:49:18.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 Processor -- start 2024-08-18T13:49:18.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 -- start start 2024-08-18T13:49:18.832 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3680 0x7fe17014ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3a50 0x7fe170158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1700c3e40 0x7fe170158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b47c0 con 0x7fe1700c3e40 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b4570 con 0x7fe1700c3a50 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700bd880 con 0x7fe1700c3680 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1700c3e40 0x7fe170158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1700c3e40 0x7fe170158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45120/0 (socket says 172.21.15.118:45120) 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 -- 172.21.15.118:0/12271511 learned_addr learned my addr 172.21.15.118:0/12271511 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16e59c700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3680 0x7fe17014ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16dd9b700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3a50 0x7fe170158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 -- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3680 msgr2=0x7fe17014ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:18.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3680 0x7fe17014ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 -- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3a50 msgr2=0x7fe170158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3a50 0x7fe170158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 -- 172.21.15.118:0/12271511 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680094f0 con 0x7fe1700c3e40 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16e59c700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3680 0x7fe17014ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16ed9d700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1700c3e40 0x7fe170158dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe168006fd0 tx=0x7fe168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe1667fc700 1 -- 172.21.15.118:0/12271511 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800bba0 con 0x7fe1700c3e40 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe1667fc700 1 -- 172.21.15.118:0/12271511 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168011630 con 0x7fe1700c3e40 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe1667fc700 1 -- 172.21.15.118:0/12271511 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1680119d0 con 0x7fe1700c3e40 2024-08-18T13:49:18.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe170152120 con 0x7fe1700c3e40 2024-08-18T13:49:18.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.826+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe170165710 con 0x7fe1700c3e40 2024-08-18T13:49:18.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.830+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1700713a0 con 0x7fe1700c3e40 2024-08-18T13:49:18.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.830+0000 7fe1667fc700 1 -- 172.21.15.118:0/12271511 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe168011d40 con 0x7fe1700c3e40 2024-08-18T13:49:18.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.830+0000 7fe1667fc700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe150060860 0x7fe150062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:18.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.830+0000 7fe16e59c700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe150060860 0x7fe150062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:18.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.830+0000 7fe1667fc700 1 -- 172.21.15.118:0/12271511 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7fe16807e810 con 0x7fe1700c3e40 2024-08-18T13:49:18.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.830+0000 7fe16e59c700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe150060860 0x7fe150062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe160001f80 tx=0x7fe160005c50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:18.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:18.838+0000 7fe1667fc700 1 -- 172.21.15.118:0/12271511 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe16804e7b0 con 0x7fe1700c3e40 2024-08-18T13:49:19.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.038+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe17014bad0 con 0x7fe1700c3e40 2024-08-18T13:49:19.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.038+0000 7fe1667fc700 1 -- 172.21.15.118:0/12271511 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fe168051d50 con 0x7fe1700c3e40 2024-08-18T13:49:19.043 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:19.044 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:19.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe150060860 msgr2=0x7fe150062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:19.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe150060860 0x7fe150062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe160001f80 tx=0x7fe160005c50 comp rx=0 tx=0).stop 2024-08-18T13:49:19.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1700c3e40 msgr2=0x7fe170158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:19.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1700c3e40 0x7fe170158dc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe168006fd0 tx=0x7fe168017000 comp rx=0 tx=0).stop 2024-08-18T13:49:19.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 shutdown_connections 2024-08-18T13:49:19.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe150060860 0x7fe150062d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:19.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1700c3e40 0x7fe170158dc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:19.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1700c3a50 0x7fe170158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:19.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 --2- 172.21.15.118:0/12271511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1700c3680 0x7fe17014ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:19.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 >> 172.21.15.118:0/12271511 conn(0x7fe17006c020 msgr2=0x7fe1700673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:19.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 shutdown_connections 2024-08-18T13:49:19.048 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:19.042+0000 7fe16f59e700 1 -- 172.21.15.118:0/12271511 wait complete. 2024-08-18T13:49:19.049 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:19.061 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:49:21.062 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:21.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda7a023700 1 Processor -- start 2024-08-18T13:49:21.182 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda7a023700 1 -- start start 2024-08-18T13:49:21.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda7a023700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda740be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:21.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda7a023700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda7415a310 0x7fda7406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:21.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda7a023700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda740c3020 con 0x7fda740c3d50 2024-08-18T13:49:21.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda7a023700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740c38c0 con 0x7fda7415a310 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda7a023700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7406e3b0 con 0x7fda740be340 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda73fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fda740c3d50 0x7fda7406b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41744/0 (socket says 172.21.15.118:41744) 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda73fff700 1 -- 172.21.15.118:0/320959930 learned_addr learned my addr 172.21.15.118:0/320959930 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda737fe700 1 --2- 172.21.15.118:0/320959930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda740be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda72ffd700 1 --2- 172.21.15.118:0/320959930 >> v2:172.21.15.178:3300/0 conn(0x7fda7415a310 0x7fda7406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda727fc700 1 -- 172.21.15.118:0/320959930 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218166625 0 0) 0x7fda740c3020 con 0x7fda740c3d50 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda727fc700 1 -- 172.21.15.118:0/320959930 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda60003570 con 0x7fda740c3d50 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda737fe700 1 -- 172.21.15.118:0/320959930 >> v2:172.21.15.178:3300/0 conn(0x7fda7415a310 msgr2=0x7fda7406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda737fe700 1 --2- 172.21.15.118:0/320959930 >> v2:172.21.15.178:3300/0 conn(0x7fda7415a310 0x7fda7406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda737fe700 1 -- 172.21.15.118:0/320959930 >> v1:172.21.15.118:6789/0 conn(0x7fda740c3d50 legacy=0x7fda7406b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:21.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda737fe700 1 -- 172.21.15.118:0/320959930 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740673d0 con 0x7fda740be340 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda737fe700 1 --2- 172.21.15.118:0/320959930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda740be710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0098e0 tx=0x7fda5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb276d0d466bd42 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda727fc700 1 -- 172.21.15.118:0/320959930 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011630 con 0x7fda740be340 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda727fc700 1 -- 172.21.15.118:0/320959930 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c011790 con 0x7fda740be340 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.178+0000 7fda727fc700 1 -- 172.21.15.118:0/320959930 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c011b10 con 0x7fda740be340 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 -- 172.21.15.118:0/320959930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 msgr2=0x7fda740be710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 --2- 172.21.15.118:0/320959930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda740be710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0098e0 tx=0x7fda5c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 -- 172.21.15.118:0/320959930 shutdown_connections 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 --2- 172.21.15.118:0/320959930 >> v2:172.21.15.178:3300/0 conn(0x7fda7415a310 0x7fda7406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 --2- 172.21.15.118:0/320959930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda740be710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 -- 172.21.15.118:0/320959930 >> 172.21.15.118:0/320959930 conn(0x7fda740c7c40 msgr2=0x7fda740c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:21.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 -- 172.21.15.118:0/320959930 shutdown_connections 2024-08-18T13:49:21.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 -- 172.21.15.118:0/320959930 wait complete. 2024-08-18T13:49:21.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 Processor -- start 2024-08-18T13:49:21.188 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.182+0000 7fda7a023700 1 -- start start 2024-08-18T13:49:21.191 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda7415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:21.191 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda740c3d50 0x7fda74169a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:21.191 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7415a310 0x7fda74169f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:21.191 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740c1780 con 0x7fda740c3d50 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7415f420 con 0x7fda7415a310 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7415f6e0 con 0x7fda740be340 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda740c3d50 0x7fda74169a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda740c3d50 0x7fda74169a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45132/0 (socket says 172.21.15.118:45132) 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 -- 172.21.15.118:0/2492302399 learned_addr learned my addr 172.21.15.118:0/2492302399 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda737fe700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda7415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda73fff700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7415a310 0x7fda74169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 -- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 msgr2=0x7fda7415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda7415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 -- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7415a310 msgr2=0x7fda74169f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7415a310 0x7fda74169f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.192 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 -- 172.21.15.118:0/2492302399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c009420 con 0x7fda740c3d50 2024-08-18T13:49:21.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda737fe700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda7415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:21.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda72ffd700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda740c3d50 0x7fda74169a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009e50 tx=0x7fda6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:21.193 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda6b7fe700 1 -- 172.21.15.118:0/2492302399 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00cc20 con 0x7fda740c3d50 2024-08-18T13:49:21.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda6b7fe700 1 -- 172.21.15.118:0/2492302399 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c00f040 con 0x7fda740c3d50 2024-08-18T13:49:21.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda740c0e50 con 0x7fda740c3d50 2024-08-18T13:49:21.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda6b7fe700 1 -- 172.21.15.118:0/2492302399 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c014480 con 0x7fda740c3d50 2024-08-18T13:49:21.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.186+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda740c1410 con 0x7fda740c3d50 2024-08-18T13:49:21.194 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.190+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda740642f0 con 0x7fda740c3d50 2024-08-18T13:49:21.195 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.190+0000 7fda6b7fe700 1 -- 172.21.15.118:0/2492302399 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fda6c0147f0 con 0x7fda740c3d50 2024-08-18T13:49:21.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.190+0000 7fda6b7fe700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda54060890 0x7fda54062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:21.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.190+0000 7fda6b7fe700 1 -- 172.21.15.118:0/2492302399 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7fda6c07dd20 con 0x7fda740c3d50 2024-08-18T13:49:21.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.190+0000 7fda737fe700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda54060890 0x7fda54062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:21.196 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.190+0000 7fda737fe700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda54060890 0x7fda54062d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0091e0 tx=0x7fda5c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:21.201 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.198+0000 7fda6b7fe700 1 -- 172.21.15.118:0/2492302399 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fda6c04dc60 con 0x7fda740c3d50 2024-08-18T13:49:21.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.398+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda74069a90 con 0x7fda740c3d50 2024-08-18T13:49:21.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda6b7fe700 1 -- 172.21.15.118:0/2492302399 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fda6c051280 con 0x7fda740c3d50 2024-08-18T13:49:21.405 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:21.405 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:21.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda54060890 msgr2=0x7fda54062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:21.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda54060890 0x7fda54062d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0091e0 tx=0x7fda5c017000 comp rx=0 tx=0).stop 2024-08-18T13:49:21.408 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda740c3d50 msgr2=0x7fda74169a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda740c3d50 0x7fda74169a40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009e50 tx=0x7fda6c00b040 comp rx=0 tx=0).stop 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 shutdown_connections 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda54060890 0x7fda54062d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda7415a310 0x7fda74169f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda740c3d50 0x7fda74169a40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 --2- 172.21.15.118:0/2492302399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda740be340 0x7fda7415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 >> 172.21.15.118:0/2492302399 conn(0x7fda740c7c40 msgr2=0x7fda740c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 shutdown_connections 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:21.402+0000 7fda7a023700 1 -- 172.21.15.118:0/2492302399 wait complete. 2024-08-18T13:49:21.409 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:21.420 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:49:23.421 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:23.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9f59e700 1 Processor -- start 2024-08-18T13:49:23.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9f59e700 1 -- start start 2024-08-18T13:49:23.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8aa014f7e0 0x7f8aa00c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:23.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa00abea0 0x7f8aa00c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:23.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa00c1c80 con 0x7f8aa014f410 2024-08-18T13:49:23.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9f59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa00c18e0 con 0x7f8aa014f7e0 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00c09f0 con 0x7f8aa00abea0 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9e59c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f8aa014f410 0x7f8aa00ab790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:41760/0 (socket says 172.21.15.118:41760) 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9e59c700 1 -- 172.21.15.118:0/1279689187 learned_addr learned my addr 172.21.15.118:0/1279689187 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9dd9b700 1 --2- 172.21.15.118:0/1279689187 >> v2:172.21.15.118:3300/0 conn(0x7f8aa014f7e0 0x7f8aa00c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9ed9d700 1 --2- 172.21.15.118:0/1279689187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa00abea0 0x7f8aa00c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9d59a700 1 -- 172.21.15.118:0/1279689187 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640752585 0 0) 0x7f8aa00c1c80 con 0x7f8aa014f410 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9d59a700 1 -- 172.21.15.118:0/1279689187 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a88003570 con 0x7f8aa014f410 2024-08-18T13:49:23.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9dd9b700 1 -- 172.21.15.118:0/1279689187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa00abea0 msgr2=0x7f8aa00c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9dd9b700 1 --2- 172.21.15.118:0/1279689187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa00abea0 0x7f8aa00c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9dd9b700 1 -- 172.21.15.118:0/1279689187 >> v1:172.21.15.118:6789/0 conn(0x7f8aa014f410 legacy=0x7f8aa00ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9dd9b700 1 -- 172.21.15.118:0/1279689187 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00c0cd0 con 0x7f8aa014f7e0 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9dd9b700 1 --2- 172.21.15.118:0/1279689187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa014f7e0 0x7f8aa00c7530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8a940098e0 tx=0x7f8a940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a75287179946dc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9d59a700 1 -- 172.21.15.118:0/1279689187 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011630 con 0x7f8aa014f7e0 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9d59a700 1 -- 172.21.15.118:0/1279689187 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94011790 con 0x7f8aa014f7e0 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.530+0000 7f8a9d59a700 1 -- 172.21.15.118:0/1279689187 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94011b30 con 0x7f8aa014f7e0 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- 172.21.15.118:0/1279689187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa014f7e0 msgr2=0x7f8aa00c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/1279689187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa014f7e0 0x7f8aa00c7530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8a940098e0 tx=0x7f8a940091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- 172.21.15.118:0/1279689187 shutdown_connections 2024-08-18T13:49:23.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/1279689187 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa00abea0 0x7f8aa00c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/1279689187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa014f7e0 0x7f8aa00c7530 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- 172.21.15.118:0/1279689187 >> 172.21.15.118:0/1279689187 conn(0x7f8aa00c8620 msgr2=0x7f8aa00c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:23.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- 172.21.15.118:0/1279689187 shutdown_connections 2024-08-18T13:49:23.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- 172.21.15.118:0/1279689187 wait complete. 2024-08-18T13:49:23.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 Processor -- start 2024-08-18T13:49:23.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- start start 2024-08-18T13:49:23.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00abea0 0x7f8aa0166500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:23.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aa014f410 0x7f8aa015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:23.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa014f7e0 0x7f8aa014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:23.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015a030 con 0x7f8aa00abea0 2024-08-18T13:49:23.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0169450 con 0x7f8aa014f410 2024-08-18T13:49:23.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.534+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01657e0 con 0x7f8aa014f7e0 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00abea0 0x7f8aa0166500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00abea0 0x7f8aa0166500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45158/0 (socket says 172.21.15.118:45158) 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 -- 172.21.15.118:0/4169177334 learned_addr learned my addr 172.21.15.118:0/4169177334 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9ed9d700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa014f7e0 0x7f8aa014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9dd9b700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aa014f410 0x7f8aa015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 -- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa014f7e0 msgr2=0x7f8aa014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa014f7e0 0x7f8aa014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 -- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aa014f410 msgr2=0x7f8aa015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aa014f410 0x7f8aa015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 -- 172.21.15.118:0/4169177334 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a94009420 con 0x7f8aa00abea0 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9e59c700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00abea0 0x7f8aa0166500 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c790 tx=0x7f8a90007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:23.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a8e7fc700 1 -- 172.21.15.118:0/4169177334 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90005b30 con 0x7f8aa00abea0 2024-08-18T13:49:23.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a8e7fc700 1 -- 172.21.15.118:0/4169177334 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90005c90 con 0x7f8aa00abea0 2024-08-18T13:49:23.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa01691b0 con 0x7f8aa00abea0 2024-08-18T13:49:23.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a8e7fc700 1 -- 172.21.15.118:0/4169177334 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90007320 con 0x7f8aa00abea0 2024-08-18T13:49:23.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa0165470 con 0x7f8aa00abea0 2024-08-18T13:49:23.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.538+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa00713a0 con 0x7f8aa00abea0 2024-08-18T13:49:23.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.542+0000 7f8a8e7fc700 1 -- 172.21.15.118:0/4169177334 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8a90007690 con 0x7f8aa00abea0 2024-08-18T13:49:23.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.542+0000 7f8a8e7fc700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a80060860 0x7f8a80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:23.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.542+0000 7f8a8e7fc700 1 -- 172.21.15.118:0/4169177334 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f8a9007c9d0 con 0x7f8aa00abea0 2024-08-18T13:49:23.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.542+0000 7f8a9dd9b700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a80060860 0x7f8a80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:23.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.542+0000 7f8a9dd9b700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a80060860 0x7f8a80062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa014fbe0 tx=0x7f8a94017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:23.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.550+0000 7f8a8e7fc700 1 -- 172.21.15.118:0/4169177334 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8a9004c8f0 con 0x7f8aa00abea0 2024-08-18T13:49:23.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.742+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8aa00c29a0 con 0x7f8aa00abea0 2024-08-18T13:49:23.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.742+0000 7f8a8e7fc700 1 -- 172.21.15.118:0/4169177334 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f8a90017090 con 0x7f8aa00abea0 2024-08-18T13:49:23.749 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:23.749 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:23.751 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a80060860 msgr2=0x7f8a80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a80060860 0x7f8a80062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa014fbe0 tx=0x7f8a94017000 comp rx=0 tx=0).stop 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00abea0 msgr2=0x7f8aa0166500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00abea0 0x7f8aa0166500 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c790 tx=0x7f8a90007df0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 shutdown_connections 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a80060860 0x7f8a80062d10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aa014f7e0 0x7f8aa014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aa014f410 0x7f8aa015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 --2- 172.21.15.118:0/4169177334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8aa00abea0 0x7f8aa0166500 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 >> 172.21.15.118:0/4169177334 conn(0x7f8aa00c8620 msgr2=0x7f8aa01571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 shutdown_connections 2024-08-18T13:49:23.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:23.746+0000 7f8a9f59e700 1 -- 172.21.15.118:0/4169177334 wait complete. 2024-08-18T13:49:23.753 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:23.762 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:49:25.763 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:25.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0d8f4700 1 Processor -- start 2024-08-18T13:49:25.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0d8f4700 1 -- start start 2024-08-18T13:49:25.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0d8f4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c081556f0 0x7f1c080c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:25.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0d8f4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c080684e0 0x7f1c08155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:25.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0d8f4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c080c1110 con 0x7f1c080be4b0 2024-08-18T13:49:25.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0d8f4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080c0b90 con 0x7f1c080684e0 2024-08-18T13:49:25.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0d8f4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080c8060 con 0x7f1c081556f0 2024-08-18T13:49:25.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0c8f2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c081556f0 0x7f1c080c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:25.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.874+0000 7f1c0c8f2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c081556f0 0x7f1c080c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:42468/0 (socket says 172.21.15.118:42468) 2024-08-18T13:49:25.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0c8f2700 1 -- 172.21.15.118:0/3424215443 learned_addr learned my addr 172.21.15.118:0/3424215443 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c07fff700 1 --2- 172.21.15.118:0/3424215443 >> v2:172.21.15.178:3300/0 conn(0x7f1c080684e0 0x7f1c08155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0c8f2700 1 -- 172.21.15.118:0/3424215443 >> v2:172.21.15.178:3300/0 conn(0x7f1c080684e0 msgr2=0x7f1c08155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0c8f2700 1 --2- 172.21.15.118:0/3424215443 >> v2:172.21.15.178:3300/0 conn(0x7f1c080684e0 0x7f1c08155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0c8f2700 1 -- 172.21.15.118:0/3424215443 >> v1:172.21.15.178:6789/0 conn(0x7f1c080be4b0 legacy=0x7f1c0814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0c8f2700 1 -- 172.21.15.118:0/3424215443 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080c8300 con 0x7f1c081556f0 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c07fff700 1 --2- 172.21.15.118:0/3424215443 >> v2:172.21.15.178:3300/0 conn(0x7f1c080684e0 0x7f1c08155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0c8f2700 1 --2- 172.21.15.118:0/3424215443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c081556f0 0x7f1c080c15a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000bc80 tx=0x7f1bf000b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b15fef475e593f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c077fe700 1 -- 172.21.15.118:0/3424215443 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf000dbb0 con 0x7f1c081556f0 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c077fe700 1 -- 172.21.15.118:0/3424215443 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf000dd10 con 0x7f1c081556f0 2024-08-18T13:49:25.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c077fe700 1 -- 172.21.15.118:0/3424215443 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf00196b0 con 0x7f1c081556f0 2024-08-18T13:49:25.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/3424215443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c081556f0 msgr2=0x7f1c080c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:25.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/3424215443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c081556f0 0x7f1c080c15a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000bc80 tx=0x7f1bf000b4f0 comp rx=0 tx=0).stop 2024-08-18T13:49:25.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/3424215443 shutdown_connections 2024-08-18T13:49:25.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/3424215443 >> v2:172.21.15.178:3300/0 conn(0x7f1c080684e0 0x7f1c08155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:25.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/3424215443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c081556f0 0x7f1c080c15a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:25.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/3424215443 >> 172.21.15.118:0/3424215443 conn(0x7f1c0806c020 msgr2=0x7f1c080c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:25.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/3424215443 shutdown_connections 2024-08-18T13:49:25.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.878+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/3424215443 wait complete. 2024-08-18T13:49:25.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 Processor -- start 2024-08-18T13:49:25.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 -- start start 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c080684e0 0x7f1c08162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c080be4b0 0x7f1c081616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c081556f0 0x7f1c08164b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08152d70 con 0x7f1c080684e0 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08148440 con 0x7f1c081556f0 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08161210 con 0x7f1c080be4b0 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0c8f2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c080684e0 0x7f1c08162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0c8f2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c080684e0 0x7f1c08162060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47064/0 (socket says 172.21.15.118:47064) 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0c8f2700 1 -- 172.21.15.118:0/1587298807 learned_addr learned my addr 172.21.15.118:0/1587298807 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d0f3700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c081556f0 0x7f1c08164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:25.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c07fff700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c080be4b0 0x7f1c081616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c07fff700 1 -- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c081556f0 msgr2=0x7f1c08164b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c07fff700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c081556f0 0x7f1c08164b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c07fff700 1 -- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c080684e0 msgr2=0x7f1c08162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c07fff700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c080684e0 0x7f1c08162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c07fff700 1 -- 172.21.15.118:0/1587298807 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf8009a20 con 0x7f1c080be4b0 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d0f3700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c081556f0 0x7f1c08164b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0c8f2700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c080684e0 0x7f1c08162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:49:25.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c07fff700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c080be4b0 0x7f1c081616a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009f30 tx=0x7f1bf800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:25.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c04ff9700 1 -- 172.21.15.118:0/1587298807 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800cb70 con 0x7f1c080be4b0 2024-08-18T13:49:25.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c04ff9700 1 -- 172.21.15.118:0/1587298807 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf800f040 con 0x7f1c080be4b0 2024-08-18T13:49:25.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf000b790 con 0x7f1c080be4b0 2024-08-18T13:49:25.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c04ff9700 1 -- 172.21.15.118:0/1587298807 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8014480 con 0x7f1c080be4b0 2024-08-18T13:49:25.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.882+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c08148690 con 0x7f1c080be4b0 2024-08-18T13:49:25.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.886+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c080713a0 con 0x7f1c080be4b0 2024-08-18T13:49:25.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.886+0000 7f1c04ff9700 1 -- 172.21.15.118:0/1587298807 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1bf80147f0 con 0x7f1c080be4b0 2024-08-18T13:49:25.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.886+0000 7f1c04ff9700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1be8060860 0x7f1be8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:25.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.886+0000 7f1c04ff9700 1 -- 172.21.15.118:0/1587298807 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f1bf807dd10 con 0x7f1c080be4b0 2024-08-18T13:49:25.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.886+0000 7f1c0c8f2700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1be8060860 0x7f1be8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:25.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.886+0000 7f1c0c8f2700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1be8060860 0x7f1be8062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000dce0 tx=0x7f1bf0019000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:25.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:25.894+0000 7f1c04ff9700 1 -- 172.21.15.118:0/1587298807 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1bf804dc50 con 0x7f1c080be4b0 2024-08-18T13:49:26.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.098+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c080be290 con 0x7f1c080be4b0 2024-08-18T13:49:26.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.102+0000 7f1c04ff9700 1 -- 172.21.15.118:0/1587298807 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f1bf8051270 con 0x7f1c080be4b0 2024-08-18T13:49:26.106 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:26.106 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:26.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1be8060860 msgr2=0x7f1be8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:26.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1be8060860 0x7f1be8062d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000dce0 tx=0x7f1bf0019000 comp rx=0 tx=0).stop 2024-08-18T13:49:26.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c080be4b0 msgr2=0x7f1c081616a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c080be4b0 0x7f1c081616a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009f30 tx=0x7f1bf800b040 comp rx=0 tx=0).stop 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 shutdown_connections 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1be8060860 0x7f1be8062d10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c081556f0 0x7f1c08164b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c080be4b0 0x7f1c081616a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 --2- 172.21.15.118:0/1587298807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c080684e0 0x7f1c08162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 >> 172.21.15.118:0/1587298807 conn(0x7f1c0806c020 msgr2=0x7f1c08155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 shutdown_connections 2024-08-18T13:49:26.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:26.106+0000 7f1c0d8f4700 1 -- 172.21.15.118:0/1587298807 wait complete. 2024-08-18T13:49:26.112 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:26.124 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:49:28.125 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:28.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.233+0000 7f684bfff700 1 Processor -- start 2024-08-18T13:49:28.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.233+0000 7f684bfff700 1 -- start start 2024-08-18T13:49:28.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f684c14f7e0 0x7f684c0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:28.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0c1c80 con 0x7f684c14f410 2024-08-18T13:49:28.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0c18e0 con 0x7f684c0abea0 2024-08-18T13:49:28.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c0c8180 con 0x7f684c14f7e0 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684affd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f684c14f410 0x7f684c0ab790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38992/0 (socket says 172.21.15.118:38992) 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684affd700 1 -- 172.21.15.118:0/4030892414 learned_addr learned my addr 172.21.15.118:0/4030892414 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684a7fc700 1 --2- 172.21.15.118:0/4030892414 >> v2:172.21.15.178:3300/0 conn(0x7f684c14f7e0 0x7f684c0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684a7fc700 1 -- 172.21.15.118:0/4030892414 >> v1:172.21.15.178:6789/0 conn(0x7f684c0abea0 legacy=0x7f684c0c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f6849ffb700 1 -- 172.21.15.118:0/4030892414 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580533723 0 0) 0x7f684c0c1c80 con 0x7f684c14f410 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684a7fc700 1 -- 172.21.15.118:0/4030892414 >> v1:172.21.15.118:6789/0 conn(0x7f684c14f410 legacy=0x7f684c0ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684a7fc700 1 -- 172.21.15.118:0/4030892414 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0c09f0 con 0x7f684c14f7e0 2024-08-18T13:49:28.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684a7fc700 1 --2- 172.21.15.118:0/4030892414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f7e0 0x7f684c0c7530 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f683c0098e0 tx=0x7f683c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54668188d9ebe9c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:28.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f6849ffb700 1 -- 172.21.15.118:0/4030892414 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c009e80 con 0x7f684c14f7e0 2024-08-18T13:49:28.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f6849ffb700 1 -- 172.21.15.118:0/4030892414 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c00b8f0 con 0x7f684c14f7e0 2024-08-18T13:49:28.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f6849ffb700 1 -- 172.21.15.118:0/4030892414 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c00bc70 con 0x7f684c14f7e0 2024-08-18T13:49:28.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- 172.21.15.118:0/4030892414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f7e0 msgr2=0x7f684c0c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:28.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 --2- 172.21.15.118:0/4030892414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f7e0 0x7f684c0c7530 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f683c0098e0 tx=0x7f683c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- 172.21.15.118:0/4030892414 shutdown_connections 2024-08-18T13:49:28.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 --2- 172.21.15.118:0/4030892414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f7e0 0x7f684c0c7530 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- 172.21.15.118:0/4030892414 >> 172.21.15.118:0/4030892414 conn(0x7f684c0c8620 msgr2=0x7f684c0c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:28.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- 172.21.15.118:0/4030892414 shutdown_connections 2024-08-18T13:49:28.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.237+0000 7f684bfff700 1 -- 172.21.15.118:0/4030892414 wait complete. 2024-08-18T13:49:28.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 Processor -- start 2024-08-18T13:49:28.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 -- start start 2024-08-18T13:49:28.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f684c0abea0 0x7f684c0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:28.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f410 0x7f684c143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:28.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f684c14f7e0 0x7f684c143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:28.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c14c7e0 con 0x7f684c14f7e0 2024-08-18T13:49:28.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c157d80 con 0x7f684c14f410 2024-08-18T13:49:28.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684bfff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c156420 con 0x7f684c0abea0 2024-08-18T13:49:28.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f684c14f7e0 0x7f684c143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:28.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f684c14f7e0 0x7f684c143a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47070/0 (socket says 172.21.15.118:47070) 2024-08-18T13:49:28.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 -- 172.21.15.118:0/4085215537 learned_addr learned my addr 172.21.15.118:0/4085215537 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:28.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684affd700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f684c0abea0 0x7f684c0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:28.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684a7fc700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f410 0x7f684c143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:28.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 -- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f684c0abea0 msgr2=0x7f684c0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:28.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f684c0abea0 0x7f684c0b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 -- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f410 msgr2=0x7f684c143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:28.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f410 0x7f684c143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 -- 172.21.15.118:0/4085215537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c009420 con 0x7f684c14f7e0 2024-08-18T13:49:28.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.241+0000 7f684b7fe700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f684c14f7e0 0x7f684c143a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f684400c910 tx=0x7f68440016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:28.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f6842ffd700 1 -- 172.21.15.118:0/4085215537 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844012630 con 0x7f684c14f7e0 2024-08-18T13:49:28.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f6842ffd700 1 -- 172.21.15.118:0/4085215537 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844012790 con 0x7f684c14f7e0 2024-08-18T13:49:28.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c0b0000 con 0x7f684c14f7e0 2024-08-18T13:49:28.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f6842ffd700 1 -- 172.21.15.118:0/4085215537 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844012b50 con 0x7f684c14f7e0 2024-08-18T13:49:28.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c0afcb0 con 0x7f684c14f7e0 2024-08-18T13:49:28.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c0b0190 con 0x7f684c14f7e0 2024-08-18T13:49:28.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f6842ffd700 1 -- 172.21.15.118:0/4085215537 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f684401b450 con 0x7f684c14f7e0 2024-08-18T13:49:28.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f6842ffd700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f682c060860 0x7f682c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:28.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f6842ffd700 1 -- 172.21.15.118:0/4085215537 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f684407dc40 con 0x7f684c14f7e0 2024-08-18T13:49:28.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f684affd700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f682c060860 0x7f682c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:28.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.245+0000 7f684affd700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f682c060860 0x7f682c062d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6834007fd0 tx=0x7f6834007d10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:28.257 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.253+0000 7f6842ffd700 1 -- 172.21.15.118:0/4085215537 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6844051690 con 0x7f684c14f7e0 2024-08-18T13:49:28.459 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.453+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f684c0713a0 con 0x7f684c14f7e0 2024-08-18T13:49:28.460 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.453+0000 7f6842ffd700 1 -- 172.21.15.118:0/4085215537 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f6844016090 con 0x7f684c14f7e0 2024-08-18T13:49:28.460 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:28.460 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5464,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:47:39.741712+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5350":{"gid":5350,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":28,"addr":"172.21.15.178:6837/1054354354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1054354354},{"type":"v1","addr":"172.21.15.178:6837","nonce":1054354354}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:28.462 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f682c060860 msgr2=0x7f682c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f682c060860 0x7f682c062d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6834007fd0 tx=0x7f6834007d10 comp rx=0 tx=0).stop 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f684c14f7e0 msgr2=0x7f684c143a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f684c14f7e0 0x7f684c143a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f684400c910 tx=0x7f68440016f0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 shutdown_connections 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f682c060860 0x7f682c062d10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f684c14f7e0 0x7f684c143a50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c14f410 0x7f684c143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 --2- 172.21.15.118:0/4085215537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f684c0abea0 0x7f684c0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 >> 172.21.15.118:0/4085215537 conn(0x7f684c0c8620 msgr2=0x7f684c0c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:28.463 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 shutdown_connections 2024-08-18T13:49:28.464 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:28.457+0000 7f684bfff700 1 -- 172.21.15.118:0/4085215537 wait complete. 2024-08-18T13:49:28.464 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 30 2024-08-18T13:49:28.473 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-08-18T13:49:30.474 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:30.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 Processor -- start 2024-08-18T13:49:30.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- start start 2024-08-18T13:49:30.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8a1c153690 0x7f8a1c0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c06ee60 con 0x7f8a1c06f2f0 2024-08-18T13:49:30.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c06f730 con 0x7f8a1c0c8200 2024-08-18T13:49:30.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1c15a640 con 0x7f8a1c153690 2024-08-18T13:49:30.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a21979700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8a1c153690 0x7f8a1c0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a21979700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f8a1c153690 0x7f8a1c0673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47076/0 (socket says 172.21.15.118:47076) 2024-08-18T13:49:30.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a21979700 1 -- 172.21.15.118:0/2634536243 learned_addr learned my addr 172.21.15.118:0/2634536243 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:30.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2217a700 1 --1- 172.21.15.118:0/2634536243 >> v1:172.21.15.118:6789/0 conn(0x7f8a1c06f2f0 0x7f8a1c0c29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38996/0 (socket says 172.21.15.118:38996) 2024-08-18T13:49:30.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a21979700 1 -- 172.21.15.118:0/2634536243 >> v1:172.21.15.178:6789/0 conn(0x7f8a1c0c8200 legacy=0x7f8a1c0c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:30.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a21979700 1 -- 172.21.15.118:0/2634536243 >> v1:172.21.15.118:6789/0 conn(0x7f8a1c06f2f0 legacy=0x7f8a1c0c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a21979700 1 -- 172.21.15.118:0/2634536243 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c153d10 con 0x7f8a1c153690 2024-08-18T13:49:30.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a21979700 1 --2- 172.21.15.118:0/2634536243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c0673d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8a100099b0 tx=0x7f8a10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3af9f328f3e45b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a20977700 1 -- 172.21.15.118:0/2634536243 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000b820 con 0x7f8a1c153690 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a20977700 1 -- 172.21.15.118:0/2634536243 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1000b980 con 0x7f8a1c153690 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a20977700 1 -- 172.21.15.118:0/2634536243 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000bd20 con 0x7f8a1c153690 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- 172.21.15.118:0/2634536243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 msgr2=0x7f8a1c0673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2634536243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c0673d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8a100099b0 tx=0x7f8a10009250 comp rx=0 tx=0).stop 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- 172.21.15.118:0/2634536243 shutdown_connections 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2634536243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c0673d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- 172.21.15.118:0/2634536243 >> 172.21.15.118:0/2634536243 conn(0x7f8a1c06c020 msgr2=0x7f8a1c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- 172.21.15.118:0/2634536243 shutdown_connections 2024-08-18T13:49:30.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.585+0000 7f8a2297b700 1 -- 172.21.15.118:0/2634536243 wait complete. 2024-08-18T13:49:30.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 Processor -- start 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 -- start start 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1c06f2f0 0x7f8a1c1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a1c0c8200 0x7f8a1c14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c157cc0 con 0x7f8a1c153690 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c16df10 con 0x7f8a1c0c8200 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c157a30 con 0x7f8a1c06f2f0 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c169e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47088/0 (socket says 172.21.15.118:47088) 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 -- 172.21.15.118:0/2465381762 learned_addr learned my addr 172.21.15.118:0/2465381762 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:30.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a21178700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a1c0c8200 0x7f8a1c14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a21979700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1c06f2f0 0x7f8a1c1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 -- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1c06f2f0 msgr2=0x7f8a1c1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1c06f2f0 0x7f8a1c1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 -- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a1c0c8200 msgr2=0x7f8a1c14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a1c0c8200 0x7f8a1c14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 -- 172.21.15.118:0/2465381762 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a100094f0 con 0x7f8a1c153690 2024-08-18T13:49:30.599 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a21979700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1c06f2f0 0x7f8a1c1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:49:30.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2217a700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c169e70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c5c0 tx=0x7f8a14007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:30.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a0dffb700 1 -- 172.21.15.118:0/2465381762 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14005a70 con 0x7f8a1c153690 2024-08-18T13:49:30.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a0dffb700 1 -- 172.21.15.118:0/2465381762 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14005bf0 con 0x7f8a1c153690 2024-08-18T13:49:30.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c16d300 con 0x7f8a1c153690 2024-08-18T13:49:30.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a0dffb700 1 -- 172.21.15.118:0/2465381762 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140070d0 con 0x7f8a1c153690 2024-08-18T13:49:30.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.589+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c16d8c0 con 0x7f8a1c153690 2024-08-18T13:49:30.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.593+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c16e050 con 0x7f8a1c153690 2024-08-18T13:49:30.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.593+0000 7f8a0dffb700 1 -- 172.21.15.118:0/2465381762 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8a14007480 con 0x7f8a1c153690 2024-08-18T13:49:30.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.593+0000 7f8a0dffb700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a00060860 0x7f8a00062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.593+0000 7f8a0dffb700 1 -- 172.21.15.118:0/2465381762 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7003+0+0 (secure 0 0 0) 0x7f8a1407c5f0 con 0x7f8a1c153690 2024-08-18T13:49:30.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.593+0000 7f8a21979700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a00060860 0x7f8a00062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.601 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.593+0000 7f8a21979700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a00060860 0x7f8a00062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a100099b0 tx=0x7f8a10016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:30.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.597+0000 7f8a0dffb700 1 -- 172.21.15.118:0/2465381762 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8a1404c530 con 0x7f8a1c153690 2024-08-18T13:49:30.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.849+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a1c0713a0 con 0x7f8a1c153690 2024-08-18T13:49:30.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.849+0000 7f8a0dffb700 1 -- 172.21.15.118:0/2465381762 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f8a1404fb50 con 0x7f8a1c153690 2024-08-18T13:49:30.855 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:30.855 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:30.622906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a00060860 msgr2=0x7f8a00062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a00060860 0x7f8a00062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a100099b0 tx=0x7f8a10016000 comp rx=0 tx=0).stop 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 msgr2=0x7f8a1c169e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c169e70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c5c0 tx=0x7f8a14007ef0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 shutdown_connections 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8a00060860 0x7f8a00062d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a1c153690 0x7f8a1c169e70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a1c0c8200 0x7f8a1c14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 --2- 172.21.15.118:0/2465381762 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8a1c06f2f0 0x7f8a1c1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 >> 172.21.15.118:0/2465381762 conn(0x7f8a1c06c020 msgr2=0x7f8a1c0c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:30.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 shutdown_connections 2024-08-18T13:49:30.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.853+0000 7f8a2297b700 1 -- 172.21.15.118:0/2465381762 wait complete. 2024-08-18T13:49:30.859 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 31 2024-08-18T13:49:30.869 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2024-08-18T13:49:30.869 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-08-18T13:49:30.869 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:30.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 Processor -- start 2024-08-18T13:49:30.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 -- start start 2024-08-18T13:49:30.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd27014bf60 0x7fd270152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd270067470 con 0x7fd27005e6b0 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd270067710 con 0x7fd2700c0c50 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd270068370 con 0x7fd27014bf60 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd274e43700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd27014bf60 0x7fd270152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd26ffff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fd27005e6b0 0x7fd27006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:39002/0 (socket says 172.21.15.118:39002) 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd26ffff700 1 -- 172.21.15.118:0/1982839163 learned_addr learned my addr 172.21.15.118:0/1982839163 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd274e43700 1 -- 172.21.15.118:0/1982839163 >> v1:172.21.15.178:6789/0 conn(0x7fd2700c0c50 legacy=0x7fd2700c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd274e43700 1 -- 172.21.15.118:0/1982839163 >> v1:172.21.15.118:6789/0 conn(0x7fd27005e6b0 legacy=0x7fd27006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd274e43700 1 -- 172.21.15.118:0/1982839163 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27006cb10 con 0x7fd27014bf60 2024-08-18T13:49:30.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd274e43700 1 --2- 172.21.15.118:0/1982839163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd27014bf60 0x7fd270152840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd2640098e0 tx=0x7fd2640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45700ce468fdb3ba server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:30.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd26f7fe700 1 -- 172.21.15.118:0/1982839163 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264009e80 con 0x7fd27014bf60 2024-08-18T13:49:30.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd26f7fe700 1 -- 172.21.15.118:0/1982839163 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26400b820 con 0x7fd27014bf60 2024-08-18T13:49:30.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd26f7fe700 1 -- 172.21.15.118:0/1982839163 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400bba0 con 0x7fd27014bf60 2024-08-18T13:49:30.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 -- 172.21.15.118:0/1982839163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd27014bf60 msgr2=0x7fd270152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 --2- 172.21.15.118:0/1982839163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd27014bf60 0x7fd270152840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd2640098e0 tx=0x7fd2640091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 -- 172.21.15.118:0/1982839163 shutdown_connections 2024-08-18T13:49:30.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 --2- 172.21.15.118:0/1982839163 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd27014bf60 0x7fd270152840 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.965+0000 7fd275e45700 1 -- 172.21.15.118:0/1982839163 >> 172.21.15.118:0/1982839163 conn(0x7fd2700c81e0 msgr2=0x7fd2700c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:30.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- 172.21.15.118:0/1982839163 shutdown_connections 2024-08-18T13:49:30.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- 172.21.15.118:0/1982839163 wait complete. 2024-08-18T13:49:30.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 Processor -- start 2024-08-18T13:49:30.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- start start 2024-08-18T13:49:30.976 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd27005e6b0 0x7fd270164160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2700c0c50 0x7fd2700c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27014bf60 0x7fd2700c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd270152d80 con 0x7fd2700c0c50 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2700c3510 con 0x7fd27014bf60 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd270153c50 con 0x7fd27005e6b0 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2700c0c50 0x7fd2700c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2700c0c50 0x7fd2700c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47110/0 (socket says 172.21.15.118:47110) 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 -- 172.21.15.118:0/661181146 learned_addr learned my addr 172.21.15.118:0/661181146 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd274e43700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd27005e6b0 0x7fd270164160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275644700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27014bf60 0x7fd2700c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 -- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd27005e6b0 msgr2=0x7fd270164160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd27005e6b0 0x7fd270164160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 -- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27014bf60 msgr2=0x7fd2700c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27014bf60 0x7fd2700c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 -- 172.21.15.118:0/661181146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd264009420 con 0x7fd2700c0c50 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26ffff700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2700c0c50 0x7fd2700c6fb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00c7c0 tx=0x7fd25c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26cff9700 1 -- 172.21.15.118:0/661181146 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c005b30 con 0x7fd2700c0c50 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26cff9700 1 -- 172.21.15.118:0/661181146 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c005c90 con 0x7fd2700c0c50 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd270153640 con 0x7fd2700c0c50 2024-08-18T13:49:30.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd26cff9700 1 -- 172.21.15.118:0/661181146 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c004a80 con 0x7fd2700c0c50 2024-08-18T13:49:30.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.969+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2700c2f50 con 0x7fd2700c0c50 2024-08-18T13:49:30.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.973+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2701534a0 con 0x7fd2700c0c50 2024-08-18T13:49:30.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.973+0000 7fd26cff9700 1 -- 172.21.15.118:0/661181146 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd25c0070d0 con 0x7fd2700c0c50 2024-08-18T13:49:30.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.973+0000 7fd26cff9700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd254060860 0x7fd254062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:30.979 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.973+0000 7fd26cff9700 1 -- 172.21.15.118:0/661181146 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fd25c07c3c0 con 0x7fd2700c0c50 2024-08-18T13:49:30.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.973+0000 7fd274e43700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd254060860 0x7fd254062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:30.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.973+0000 7fd274e43700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd254060860 0x7fd254062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd2640030c0 tx=0x7fd264016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:30.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:30.981+0000 7fd26cff9700 1 -- 172.21.15.118:0/661181146 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd25c04a480 con 0x7fd2700c0c50 2024-08-18T13:49:31.183 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.177+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd270153dc0 con 0x7fd2700c0c50 2024-08-18T13:49:31.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.177+0000 7fd26cff9700 1 -- 172.21.15.118:0/661181146 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5369 (secure 0 0 0) 0x7fd2701534a0 con 0x7fd2700c0c50 2024-08-18T13:49:31.184 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:31.184 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:30.622906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:31.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd254060860 msgr2=0x7fd254062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:31.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd254060860 0x7fd254062d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd2640030c0 tx=0x7fd264016000 comp rx=0 tx=0).stop 2024-08-18T13:49:31.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2700c0c50 msgr2=0x7fd2700c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:31.186 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2700c0c50 0x7fd2700c6fb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00c7c0 tx=0x7fd25c007f70 comp rx=0 tx=0).stop 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 shutdown_connections 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd254060860 0x7fd254062d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27014bf60 0x7fd2700c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2700c0c50 0x7fd2700c6fb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 --2- 172.21.15.118:0/661181146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd27005e6b0 0x7fd270164160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 >> 172.21.15.118:0/661181146 conn(0x7fd2700c81e0 msgr2=0x7fd2700c1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 shutdown_connections 2024-08-18T13:49:31.187 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:31.181+0000 7fd275e45700 1 -- 172.21.15.118:0/661181146 wait complete. 2024-08-18T13:49:31.188 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 31 2024-08-18T13:49:31.199 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 31, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 31, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5464": { "addr": "172.21.15.178:6835/2054758182", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6832", "nonce": 2054758182, "type": "v2" }, { "addr": "172.21.15.178:6835", "nonce": 2054758182, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5464, "incarnation": 31, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:49:30.622906+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5464 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:49:33.200 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:33.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.305+0000 7f833441d700 1 Processor -- start 2024-08-18T13:49:33.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.305+0000 7f833441d700 1 -- start start 2024-08-18T13:49:33.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f833441d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c153100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:33.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f833441d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f832c05e760 0x7f832c0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f833441d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f832c0c4910 0x7f832c14cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f833441d700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0c3dd0 con 0x7f832c05e760 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f833441d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c1540a0 con 0x7f832c0c4910 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f833441d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c5480 con 0x7f832c0c4540 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f832c05e760 0x7f832c0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f832c05e760 0x7f832c0c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47118/0 (socket says 172.21.15.118:47118) 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 -- 172.21.15.118:0/3277888078 learned_addr learned my addr 172.21.15.118:0/3277888078 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:33.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83329ba700 1 --2- 172.21.15.118:0/3277888078 >> v2:172.21.15.178:3300/0 conn(0x7f832c0c4910 0x7f832c14cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83321b9700 1 --2- 172.21.15.118:0/3277888078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 -- 172.21.15.118:0/3277888078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 msgr2=0x7f832c153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 --2- 172.21.15.118:0/3277888078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 -- 172.21.15.118:0/3277888078 >> v2:172.21.15.178:3300/0 conn(0x7f832c0c4910 msgr2=0x7f832c14cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 --2- 172.21.15.118:0/3277888078 >> v2:172.21.15.178:3300/0 conn(0x7f832c0c4910 0x7f832c14cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 -- 172.21.15.118:0/3277888078 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0c2710 con 0x7f832c05e760 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83321b9700 1 --2- 172.21.15.118:0/3277888078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83319b8700 1 --2- 172.21.15.118:0/3277888078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c05e760 0x7f832c0c5910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f832c068f10 tx=0x7f831c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657870857666876b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83311b7700 1 -- 172.21.15.118:0/3277888078 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c009ea0 con 0x7f832c05e760 2024-08-18T13:49:33.318 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83311b7700 1 -- 172.21.15.118:0/3277888078 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00b820 con 0x7f832c05e760 2024-08-18T13:49:33.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.309+0000 7f83311b7700 1 -- 172.21.15.118:0/3277888078 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00bbc0 con 0x7f832c05e760 2024-08-18T13:49:33.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 -- 172.21.15.118:0/3277888078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c05e760 msgr2=0x7f832c0c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:33.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 --2- 172.21.15.118:0/3277888078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c05e760 0x7f832c0c5910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f832c068f10 tx=0x7f831c009250 comp rx=0 tx=0).stop 2024-08-18T13:49:33.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 -- 172.21.15.118:0/3277888078 shutdown_connections 2024-08-18T13:49:33.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 --2- 172.21.15.118:0/3277888078 >> v2:172.21.15.178:3300/0 conn(0x7f832c0c4910 0x7f832c14cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 --2- 172.21.15.118:0/3277888078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c05e760 0x7f832c0c5910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 --2- 172.21.15.118:0/3277888078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 -- 172.21.15.118:0/3277888078 >> 172.21.15.118:0/3277888078 conn(0x7f832c06bf40 msgr2=0x7f832c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:33.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 -- 172.21.15.118:0/3277888078 shutdown_connections 2024-08-18T13:49:33.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 -- 172.21.15.118:0/3277888078 wait complete. 2024-08-18T13:49:33.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 Processor -- start 2024-08-18T13:49:33.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.313+0000 7f833441d700 1 -- start start 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c05e760 0x7f832c151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c0c4910 0x7f832c147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c1573d0 con 0x7f832c0c4910 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c1554b0 con 0x7f832c05e760 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c155d50 con 0x7f832c0c4540 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c0c4910 0x7f832c147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c0c4910 0x7f832c147a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:47128/0 (socket says 172.21.15.118:47128) 2024-08-18T13:49:33.325 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 -- 172.21.15.118:0/444020285 learned_addr learned my addr 172.21.15.118:0/444020285 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83321b9700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c05e760 0x7f832c151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83319b8700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 -- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 msgr2=0x7f832c147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 -- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c05e760 msgr2=0x7f832c151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c05e760 0x7f832c151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 -- 172.21.15.118:0/444020285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0094f0 con 0x7f832c0c4910 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83321b9700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c05e760 0x7f832c151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83329ba700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c0c4910 0x7f832c147a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8324009e20 tx=0x7f832400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83227fc700 1 -- 172.21.15.118:0/444020285 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400cc20 con 0x7f832c0c4910 2024-08-18T13:49:33.326 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83227fc700 1 -- 172.21.15.118:0/444020285 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832400f040 con 0x7f832c0c4910 2024-08-18T13:49:33.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83227fc700 1 -- 172.21.15.118:0/444020285 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324014480 con 0x7f832c0c4910 2024-08-18T13:49:33.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c1662c0 con 0x7f832c0c4910 2024-08-18T13:49:33.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c1559e0 con 0x7f832c0c4910 2024-08-18T13:49:33.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f83227fc700 1 -- 172.21.15.118:0/444020285 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8324014830 con 0x7f832c0c4910 2024-08-18T13:49:33.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.317+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c068350 con 0x7f832c0c4910 2024-08-18T13:49:33.327 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.321+0000 7f83227fc700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8310060860 0x7f8310062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:33.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.321+0000 7f83227fc700 1 -- 172.21.15.118:0/444020285 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7f832407db50 con 0x7f832c0c4910 2024-08-18T13:49:33.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.321+0000 7f83321b9700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8310060860 0x7f8310062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:33.328 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.321+0000 7f83321b9700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8310060860 0x7f8310062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8328006f00 tx=0x7f8328008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:33.330 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.325+0000 7f83227fc700 1 -- 172.21.15.118:0/444020285 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f832c068350 con 0x7f832c0c4910 2024-08-18T13:49:33.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.529+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832c068350 con 0x7f832c0c4910 2024-08-18T13:49:33.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.529+0000 7f83227fc700 1 -- 172.21.15.118:0/444020285 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f832c068350 con 0x7f832c0c4910 2024-08-18T13:49:33.535 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:33.535 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:30.622906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.529+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8310060860 msgr2=0x7f8310062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.529+0000 7f833441d700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8310060860 0x7f8310062d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8328006f00 tx=0x7f8328008040 comp rx=0 tx=0).stop 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.529+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c0c4910 msgr2=0x7f832c147a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.529+0000 7f833441d700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c0c4910 0x7f832c147a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8324009e20 tx=0x7f832400b040 comp rx=0 tx=0).stop 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 shutdown_connections 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8310060860 0x7f8310062d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f832c0c4910 0x7f832c147a70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c0c4540 0x7f832c147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 --2- 172.21.15.118:0/444020285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c05e760 0x7f832c151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 >> 172.21.15.118:0/444020285 conn(0x7f832c06bf40 msgr2=0x7f832c14cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:33.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 shutdown_connections 2024-08-18T13:49:33.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:33.533+0000 7f833441d700 1 -- 172.21.15.118:0/444020285 wait complete. 2024-08-18T13:49:33.539 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 31 2024-08-18T13:49:33.549 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:35.550 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:35.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 Processor -- start 2024-08-18T13:49:35.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 -- start start 2024-08-18T13:49:35.661 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd88414bf60 0x7fd884152840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:35.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd88405e6b0 0x7fd88406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:35.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 0x7fd8840c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd884068370 con 0x7fd88405e6b0 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd884152d80 con 0x7fd88414bf60 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd883fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd884067470 con 0x7fd88406c1e0 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.653+0000 7fd87bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd88405e6b0 0x7fd88406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd88405e6b0 0x7fd88406bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48758/0 (socket says 172.21.15.118:48758) 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 -- 172.21.15.118:0/2687016737 learned_addr learned my addr 172.21.15.118:0/2687016737 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 -- 172.21.15.118:0/2687016737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 msgr2=0x7fd8840c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 --2- 172.21.15.118:0/2687016737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 0x7fd8840c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 -- 172.21.15.118:0/2687016737 >> v2:172.21.15.178:3300/0 conn(0x7fd88414bf60 msgr2=0x7fd884152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 --2- 172.21.15.118:0/2687016737 >> v2:172.21.15.178:3300/0 conn(0x7fd88414bf60 0x7fd884152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 -- 172.21.15.118:0/2687016737 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884067710 con 0x7fd88405e6b0 2024-08-18T13:49:35.664 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 --2- 172.21.15.118:0/2687016737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88406bca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd88406ec80 tx=0x7fd86c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1087ad23b4d1895e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd8827fc700 1 -- 172.21.15.118:0/2687016737 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00b8f0 con 0x7fd88405e6b0 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd8827fc700 1 -- 172.21.15.118:0/2687016737 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c00ba50 con 0x7fd88405e6b0 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd8827fc700 1 -- 172.21.15.118:0/2687016737 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00bdf0 con 0x7fd88405e6b0 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- 172.21.15.118:0/2687016737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 msgr2=0x7fd88406bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 --2- 172.21.15.118:0/2687016737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88406bca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd88406ec80 tx=0x7fd86c009250 comp rx=0 tx=0).stop 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- 172.21.15.118:0/2687016737 shutdown_connections 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 --2- 172.21.15.118:0/2687016737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 0x7fd8840c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 --2- 172.21.15.118:0/2687016737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88406bca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 --2- 172.21.15.118:0/2687016737 >> v2:172.21.15.178:3300/0 conn(0x7fd88414bf60 0x7fd884152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- 172.21.15.118:0/2687016737 >> 172.21.15.118:0/2687016737 conn(0x7fd8840c81e0 msgr2=0x7fd8840c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- 172.21.15.118:0/2687016737 shutdown_connections 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- 172.21.15.118:0/2687016737 wait complete. 2024-08-18T13:49:35.665 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 Processor -- start 2024-08-18T13:49:35.666 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- start start 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88414a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 0x7fd88414a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88414bf60 0x7fd884154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88414d750 con 0x7fd88405e6b0 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884156080 con 0x7fd88414bf60 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd883fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd884163ed0 con 0x7fd88406c1e0 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd882ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88414a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd882ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88414a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48764/0 (socket says 172.21.15.118:48764) 2024-08-18T13:49:35.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd882ffd700 1 -- 172.21.15.118:0/1655065494 learned_addr learned my addr 172.21.15.118:0/1655065494 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd87bfff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 0x7fd88414a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.657+0000 7fd8837fe700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88414bf60 0x7fd884154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd882ffd700 1 -- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 msgr2=0x7fd88414a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd882ffd700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 0x7fd88414a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd882ffd700 1 -- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88414bf60 msgr2=0x7fd884154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd882ffd700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88414bf60 0x7fd884154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd882ffd700 1 -- 172.21.15.118:0/1655065494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c0094f0 con 0x7fd88405e6b0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd882ffd700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88414a460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd87400b590 tx=0x7fd874006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87affd700 1 -- 172.21.15.118:0/1655065494 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87400f510 con 0x7fd88405e6b0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87affd700 1 -- 172.21.15.118:0/1655065494 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87400f670 con 0x7fd88405e6b0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8841641b0 con 0x7fd88405e6b0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87affd700 1 -- 172.21.15.118:0/1655065494 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87400fa30 con 0x7fd88405e6b0 2024-08-18T13:49:35.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88414b540 con 0x7fd88405e6b0 2024-08-18T13:49:35.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8840c1ff0 con 0x7fd88405e6b0 2024-08-18T13:49:35.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87affd700 1 -- 172.21.15.118:0/1655065494 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd87401a400 con 0x7fd88405e6b0 2024-08-18T13:49:35.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87affd700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd864060860 0x7fd864062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:35.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87affd700 1 -- 172.21.15.118:0/1655065494 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fd87407cb90 con 0x7fd88405e6b0 2024-08-18T13:49:35.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87bfff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd864060860 0x7fd864062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:35.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.661+0000 7fd87bfff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd864060860 0x7fd864062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd88406ec80 tx=0x7fd86c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:35.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.669+0000 7fd87affd700 1 -- 172.21.15.118:0/1655065494 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd8840c1ff0 con 0x7fd88405e6b0 2024-08-18T13:49:35.870 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.865+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8840c1ff0 con 0x7fd88405e6b0 2024-08-18T13:49:35.871 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.865+0000 7fd87affd700 1 -- 172.21.15.118:0/1655065494 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5369 (secure 0 0 0) 0x7fd8840c1ff0 con 0x7fd88405e6b0 2024-08-18T13:49:35.871 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:35.871 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:30.622906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd864060860 msgr2=0x7fd864062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd864060860 0x7fd864062d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd88406ec80 tx=0x7fd86c017000 comp rx=0 tx=0).stop 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 msgr2=0x7fd88414a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88414a460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd87400b590 tx=0x7fd874006f90 comp rx=0 tx=0).stop 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 shutdown_connections 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd864060860 0x7fd864062d10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88414bf60 0x7fd884154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd88406c1e0 0x7fd88414a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 --2- 172.21.15.118:0/1655065494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd88405e6b0 0x7fd88414a460 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 >> 172.21.15.118:0/1655065494 conn(0x7fd8840c81e0 msgr2=0x7fd8841468f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 shutdown_connections 2024-08-18T13:49:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:35.869+0000 7fd883fff700 1 -- 172.21.15.118:0/1655065494 wait complete. 2024-08-18T13:49:35.875 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 31 2024-08-18T13:49:35.885 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:37.886 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:38.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f15679ad700 1 Processor -- start 2024-08-18T13:49:38.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f15679ad700 1 -- start start 2024-08-18T13:49:38.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f15679ad700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f15600686c0 0x7f1560155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:38.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f15679ad700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1560151720 con 0x7f156006db20 2024-08-18T13:49:38.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f15679ad700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15601519c0 con 0x7f156014b070 2024-08-18T13:49:38.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f15679ad700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1560151530 con 0x7f15600686c0 2024-08-18T13:49:38.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1564f48700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f15600686c0 0x7f1560155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:38.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1564f48700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f15600686c0 0x7f1560155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48772/0 (socket says 172.21.15.118:48772) 2024-08-18T13:49:38.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1564f48700 1 -- 172.21.15.118:0/1522419970 learned_addr learned my addr 172.21.15.118:0/1522419970 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:38.004 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1565749700 1 --1- 172.21.15.118:0/1522419970 >> v1:172.21.15.118:6789/0 conn(0x7f156006db20 0x7f156006b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:52136/0 (socket says 172.21.15.118:52136) 2024-08-18T13:49:38.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1564f48700 1 -- 172.21.15.118:0/1522419970 >> v1:172.21.15.178:6789/0 conn(0x7f156014b070 legacy=0x7f156014b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:38.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1564f48700 1 -- 172.21.15.118:0/1522419970 >> v1:172.21.15.118:6789/0 conn(0x7f156006db20 legacy=0x7f156006b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:38.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1564f48700 1 -- 172.21.15.118:0/1522419970 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156014bbe0 con 0x7f15600686c0 2024-08-18T13:49:38.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1564f48700 1 --2- 172.21.15.118:0/1522419970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15600686c0 0x7f1560155510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f155400d1a0 tx=0x7f155400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d314aa5098aae8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:38.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1553fff700 1 -- 172.21.15.118:0/1522419970 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554010070 con 0x7f15600686c0 2024-08-18T13:49:38.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1553fff700 1 -- 172.21.15.118:0/1522419970 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155400b0d0 con 0x7f15600686c0 2024-08-18T13:49:38.005 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:37.997+0000 7f1553fff700 1 -- 172.21.15.118:0/1522419970 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155400b490 con 0x7f15600686c0 2024-08-18T13:49:38.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.001+0000 7f15679ad700 1 -- 172.21.15.118:0/1522419970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15600686c0 msgr2=0x7f1560155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:38.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.001+0000 7f15679ad700 1 --2- 172.21.15.118:0/1522419970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15600686c0 0x7f1560155510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f155400d1a0 tx=0x7f155400ba50 comp rx=0 tx=0).stop 2024-08-18T13:49:38.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.001+0000 7f15679ad700 1 -- 172.21.15.118:0/1522419970 shutdown_connections 2024-08-18T13:49:38.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.001+0000 7f15679ad700 1 --2- 172.21.15.118:0/1522419970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f15600686c0 0x7f1560155510 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:38.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.001+0000 7f15679ad700 1 -- 172.21.15.118:0/1522419970 >> 172.21.15.118:0/1522419970 conn(0x7f15600c7c40 msgr2=0x7f15600c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:38.006 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.001+0000 7f15679ad700 1 -- 172.21.15.118:0/1522419970 shutdown_connections 2024-08-18T13:49:38.007 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.001+0000 7f15679ad700 1 -- 172.21.15.118:0/1522419970 wait complete. 2024-08-18T13:49:38.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 Processor -- start 2024-08-18T13:49:38.009 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 -- start start 2024-08-18T13:49:38.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156006db20 0x7f1560153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:38.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156014b070 0x7f1560152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:38.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15600be750 0x7f1560153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560160e30 con 0x7f156014b070 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601610d0 con 0x7f15600be750 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15601613b0 con 0x7f156006db20 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1564f48700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156014b070 0x7f1560152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156006db20 0x7f1560153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156006db20 0x7f1560153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:45914/0 (socket says 172.21.15.118:45914) 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1564f48700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156014b070 0x7f1560152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48788/0 (socket says 172.21.15.118:48788) 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565f4a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15600be750 0x7f1560153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:38.012 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 -- 172.21.15.118:0/580561838 learned_addr learned my addr 172.21.15.118:0/580561838 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:38.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 -- 172.21.15.118:0/580561838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15600be750 msgr2=0x7f1560153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:38.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15600be750 0x7f1560153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:38.013 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 -- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156014b070 msgr2=0x7f1560152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156014b070 0x7f1560152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 -- 172.21.15.118:0/580561838 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155400bcf0 con 0x7f156006db20 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f1565749700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156006db20 0x7f1560153e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c510 tx=0x7f155c007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15517fa700 1 -- 172.21.15.118:0/580561838 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c005a70 con 0x7f156006db20 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15517fa700 1 -- 172.21.15.118:0/580561838 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c005bd0 con 0x7f156006db20 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1560161690 con 0x7f156006db20 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15517fa700 1 -- 172.21.15.118:0/580561838 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0070d0 con 0x7f156006db20 2024-08-18T13:49:38.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.005+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156006e9c0 con 0x7f156006db20 2024-08-18T13:49:38.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.009+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1560161b10 con 0x7f156006db20 2024-08-18T13:49:38.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.009+0000 7f15517fa700 1 -- 172.21.15.118:0/580561838 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f155c007480 con 0x7f156006db20 2024-08-18T13:49:38.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.009+0000 7f15517fa700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1544060910 0x7f1544062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:38.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.009+0000 7f1564f48700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1544060910 0x7f1544062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:38.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.009+0000 7f15517fa700 1 -- 172.21.15.118:0/580561838 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7f155c07c4f0 con 0x7f156006db20 2024-08-18T13:49:38.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.009+0000 7f1564f48700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1544060910 0x7f1544062dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1560068a90 tx=0x7f155401a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:38.023 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.017+0000 7f15517fa700 1 -- 172.21.15.118:0/580561838 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f155c04c330 con 0x7f156006db20 2024-08-18T13:49:38.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.221+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15601614f0 con 0x7f156006db20 2024-08-18T13:49:38.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.221+0000 7f15517fa700 1 -- 172.21.15.118:0/580561838 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f15601614f0 con 0x7f156006db20 2024-08-18T13:49:38.229 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:38.229 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:30.622906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:38.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.225+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1544060910 msgr2=0x7f1544062dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:38.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.225+0000 7f15679ad700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1544060910 0x7f1544062dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1560068a90 tx=0x7f155401a000 comp rx=0 tx=0).stop 2024-08-18T13:49:38.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.225+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156006db20 msgr2=0x7f1560153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:38.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.225+0000 7f15679ad700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156006db20 0x7f1560153e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c510 tx=0x7f155c007f70 comp rx=0 tx=0).stop 2024-08-18T13:49:38.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.225+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 shutdown_connections 2024-08-18T13:49:38.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.225+0000 7f15679ad700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1544060910 0x7f1544062dc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:38.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.225+0000 7f15679ad700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15600be750 0x7f1560153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:38.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.229+0000 7f15679ad700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f156014b070 0x7f1560152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:38.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.229+0000 7f15679ad700 1 --2- 172.21.15.118:0/580561838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156006db20 0x7f1560153e70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:38.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.229+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 >> 172.21.15.118:0/580561838 conn(0x7f15600c7c40 msgr2=0x7f156006b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:38.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.229+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 shutdown_connections 2024-08-18T13:49:38.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:38.229+0000 7f15679ad700 1 -- 172.21.15.118:0/580561838 wait complete. 2024-08-18T13:49:38.235 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 31 2024-08-18T13:49:38.247 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:40.247 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:40.371 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae732d0700 1 Processor -- start 2024-08-18T13:49:40.371 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae732d0700 1 -- start start 2024-08-18T13:49:40.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae732d0700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fae6c0c3a50 0x7fae6c14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:40.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae732d0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6c06cc90 con 0x7fae6c0c3e70 2024-08-18T13:49:40.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae732d0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6c0c83a0 con 0x7fae6c0c3680 2024-08-18T13:49:40.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae732d0700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6c067ae0 con 0x7fae6c0c3a50 2024-08-18T13:49:40.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae71acd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fae6c0c3a50 0x7fae6c14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:40.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae71acd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fae6c0c3a50 0x7fae6c14cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48794/0 (socket says 172.21.15.118:48794) 2024-08-18T13:49:40.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.365+0000 7fae71acd700 1 -- 172.21.15.118:0/3137398002 learned_addr learned my addr 172.21.15.118:0/3137398002 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:40.373 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae71acd700 1 -- 172.21.15.118:0/3137398002 >> v1:172.21.15.178:6789/0 conn(0x7fae6c0c3680 legacy=0x7fae6c0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:40.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae71acd700 1 -- 172.21.15.118:0/3137398002 >> v1:172.21.15.118:6789/0 conn(0x7fae6c0c3e70 legacy=0x7fae6c14d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:40.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae71acd700 1 -- 172.21.15.118:0/3137398002 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c153090 con 0x7fae6c0c3a50 2024-08-18T13:49:40.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae71acd700 1 --2- 172.21.15.118:0/3137398002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3a50 0x7fae6c14cd00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae680099b0 tx=0x7fae68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce40aef0b83d55f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:40.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae712cc700 1 -- 172.21.15.118:0/3137398002 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800b860 con 0x7fae6c0c3a50 2024-08-18T13:49:40.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae712cc700 1 -- 172.21.15.118:0/3137398002 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6800b9c0 con 0x7fae6c0c3a50 2024-08-18T13:49:40.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae712cc700 1 -- 172.21.15.118:0/3137398002 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800bd60 con 0x7fae6c0c3a50 2024-08-18T13:49:40.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 -- 172.21.15.118:0/3137398002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3a50 msgr2=0x7fae6c14cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:40.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 --2- 172.21.15.118:0/3137398002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3a50 0x7fae6c14cd00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae680099b0 tx=0x7fae68009250 comp rx=0 tx=0).stop 2024-08-18T13:49:40.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 -- 172.21.15.118:0/3137398002 shutdown_connections 2024-08-18T13:49:40.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 --2- 172.21.15.118:0/3137398002 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3a50 0x7fae6c14cd00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 -- 172.21.15.118:0/3137398002 >> 172.21.15.118:0/3137398002 conn(0x7fae6c06c020 msgr2=0x7fae6c0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:40.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 -- 172.21.15.118:0/3137398002 shutdown_connections 2024-08-18T13:49:40.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 -- 172.21.15.118:0/3137398002 wait complete. 2024-08-18T13:49:40.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 Processor -- start 2024-08-18T13:49:40.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.369+0000 7fae732d0700 1 -- start start 2024-08-18T13:49:40.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3680 0x7fae6c163c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:40.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae6c0c3a50 0x7fae6c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae6c0c3e70 0x7fae6c14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c14f890 con 0x7fae6c0c3680 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c14bd90 con 0x7fae6c0c3a50 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c14fb30 con 0x7fae6c0c3e70 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3680 0x7fae6c163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3680 0x7fae6c163c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48808/0 (socket says 172.21.15.118:48808) 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 -- 172.21.15.118:0/3811894571 learned_addr learned my addr 172.21.15.118:0/3811894571 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae72acf700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae6c0c3e70 0x7fae6c14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae71acd700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae6c0c3a50 0x7fae6c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 -- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae6c0c3e70 msgr2=0x7fae6c14d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae6c0c3e70 0x7fae6c14d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 -- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae6c0c3a50 msgr2=0x7fae6c155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:40.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae6c0c3a50 0x7fae6c155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 -- 172.21.15.118:0/3811894571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae680094f0 con 0x7fae6c0c3680 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae72acf700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae6c0c3e70 0x7fae6c14d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae71acd700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae6c0c3a50 0x7fae6c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae722ce700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3680 0x7fae6c163c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c4e0 tx=0x7fae5c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae627fc700 1 -- 172.21.15.118:0/3811894571 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c011590 con 0x7fae6c0c3680 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae627fc700 1 -- 172.21.15.118:0/3811894571 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c011710 con 0x7fae6c0c3680 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c14fe10 con 0x7fae6c0c3680 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae627fc700 1 -- 172.21.15.118:0/3811894571 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c011ad0 con 0x7fae6c0c3680 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c14b230 con 0x7fae6c0c3680 2024-08-18T13:49:40.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.373+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c0713a0 con 0x7fae6c0c3680 2024-08-18T13:49:40.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.377+0000 7fae627fc700 1 -- 172.21.15.118:0/3811894571 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fae5c01b3b0 con 0x7fae6c0c3680 2024-08-18T13:49:40.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.377+0000 7fae627fc700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae50060890 0x7fae50062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:40.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.377+0000 7fae71acd700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae50060890 0x7fae50062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:40.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.377+0000 7fae627fc700 1 -- 172.21.15.118:0/3811894571 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fae5c07c670 con 0x7fae6c0c3680 2024-08-18T13:49:40.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.377+0000 7fae71acd700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae50060890 0x7fae50062d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae680099b0 tx=0x7fae68017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:40.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.381+0000 7fae627fc700 1 -- 172.21.15.118:0/3811894571 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fae5c04c550 con 0x7fae6c0c3680 2024-08-18T13:49:40.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.581+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae6c150980 con 0x7fae6c0c3680 2024-08-18T13:49:40.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.581+0000 7fae627fc700 1 -- 172.21.15.118:0/3811894571 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5373 (secure 0 0 0) 0x7fae5c04fb70 con 0x7fae6c0c3680 2024-08-18T13:49:40.588 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:40.588 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:38.654010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae50060890 msgr2=0x7fae50062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae50060890 0x7fae50062d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae680099b0 tx=0x7fae68017000 comp rx=0 tx=0).stop 2024-08-18T13:49:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3680 msgr2=0x7fae6c163c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3680 0x7fae6c163c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c4e0 tx=0x7fae5c00c8a0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 shutdown_connections 2024-08-18T13:49:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fae50060890 0x7fae50062d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae6c0c3e70 0x7fae6c14d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae6c0c3a50 0x7fae6c155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 --2- 172.21.15.118:0/3811894571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae6c0c3680 0x7fae6c163c40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 >> 172.21.15.118:0/3811894571 conn(0x7fae6c06c020 msgr2=0x7fae6c14c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 shutdown_connections 2024-08-18T13:49:40.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:40.585+0000 7fae732d0700 1 -- 172.21.15.118:0/3811894571 wait complete. 2024-08-18T13:49:40.592 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 32 2024-08-18T13:49:40.602 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:42.603 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:42.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515359e700 1 Processor -- start 2024-08-18T13:49:42.716 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515359e700 1 -- start start 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515359e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f51540c3a50 0x7f515414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515359e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515406cc90 con 0x7f51540c3e40 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515359e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51540c83a0 con 0x7f51540c3a50 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5154153090 con 0x7f51540c3680 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f5151d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f51540c3a50 0x7f515414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f5151d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f51540c3a50 0x7f515414cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48816/0 (socket says 172.21.15.118:48816) 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f5151d9b700 1 -- 172.21.15.118:0/1831114125 learned_addr learned my addr 172.21.15.118:0/1831114125 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:42.717 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.709+0000 7f515259c700 1 --2- 172.21.15.118:0/1831114125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f5151d9b700 1 -- 172.21.15.118:0/1831114125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 msgr2=0x7f51540673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f5151d9b700 1 --2- 172.21.15.118:0/1831114125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f5151d9b700 1 -- 172.21.15.118:0/1831114125 >> v1:172.21.15.118:6789/0 conn(0x7f51540c3e40 legacy=0x7f515414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515159a700 1 -- 172.21.15.118:0/1831114125 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846638718 0 0) 0x7f515406cc90 con 0x7f51540c3e40 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f5151d9b700 1 -- 172.21.15.118:0/1831114125 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540c8660 con 0x7f51540c3a50 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f5151d9b700 1 --2- 172.21.15.118:0/1831114125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3a50 0x7f515414cd00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5144009f20 tx=0x7f5144009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1163e81926d18c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515159a700 1 -- 172.21.15.118:0/1831114125 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400e070 con 0x7f51540c3a50 2024-08-18T13:49:42.719 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515159a700 1 -- 172.21.15.118:0/1831114125 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514400b9d0 con 0x7f51540c3a50 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515159a700 1 -- 172.21.15.118:0/1831114125 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440092e0 con 0x7f51540c3a50 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 -- 172.21.15.118:0/1831114125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3a50 msgr2=0x7f515414cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 --2- 172.21.15.118:0/1831114125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3a50 0x7f515414cd00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5144009f20 tx=0x7f5144009840 comp rx=0 tx=0).stop 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 -- 172.21.15.118:0/1831114125 shutdown_connections 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 --2- 172.21.15.118:0/1831114125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3a50 0x7f515414cd00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 --2- 172.21.15.118:0/1831114125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 -- 172.21.15.118:0/1831114125 >> 172.21.15.118:0/1831114125 conn(0x7f515406c020 msgr2=0x7f51540c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 -- 172.21.15.118:0/1831114125 shutdown_connections 2024-08-18T13:49:42.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 -- 172.21.15.118:0/1831114125 wait complete. 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 Processor -- start 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.713+0000 7f515359e700 1 -- start start 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51540c3a50 0x7f51541423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3e40 0x7f5154142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5154148870 con 0x7f51540c3e40 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5154152100 con 0x7f51540c3a50 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5154148d80 con 0x7f51540c3680 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3e40 0x7f5154142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3e40 0x7f5154142920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:48824/0 (socket says 172.21.15.118:48824) 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 -- 172.21.15.118:0/182198537 learned_addr learned my addr 172.21.15.118:0/182198537 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:42.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515259c700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5151d9b700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51540c3a50 0x7f51541423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 -- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 msgr2=0x7f51540c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 -- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51540c3a50 msgr2=0x7f51541423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51540c3a50 0x7f51541423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 -- 172.21.15.118:0/182198537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144009ae0 con 0x7f51540c3e40 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f5152d9d700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3e40 0x7f5154142920 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c510 tx=0x7f514c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f514a7fc700 1 -- 172.21.15.118:0/182198537 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c011590 con 0x7f51540c3e40 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f514a7fc700 1 -- 172.21.15.118:0/182198537 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c0116f0 con 0x7f51540c3e40 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f514a7fc700 1 -- 172.21.15.118:0/182198537 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c011ab0 con 0x7f51540c3e40 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5154162c40 con 0x7f51540c3e40 2024-08-18T13:49:42.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51540c5d20 con 0x7f51540c3e40 2024-08-18T13:49:42.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5154162e30 con 0x7f51540c3e40 2024-08-18T13:49:42.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.717+0000 7f514a7fc700 1 -- 172.21.15.118:0/182198537 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f514c01b3b0 con 0x7f51540c3e40 2024-08-18T13:49:42.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.721+0000 7f514a7fc700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5134060860 0x7f5134062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:42.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.721+0000 7f514a7fc700 1 -- 172.21.15.118:0/182198537 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7f514c07c510 con 0x7f51540c3e40 2024-08-18T13:49:42.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.721+0000 7f515259c700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5134060860 0x7f5134062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:42.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.721+0000 7f515259c700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5134060860 0x7f5134062d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f513c009d10 tx=0x7f513c0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:42.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.729+0000 7f514a7fc700 1 -- 172.21.15.118:0/182198537 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f514c04c3f0 con 0x7f51540c3e40 2024-08-18T13:49:42.933 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51540713a0 con 0x7f51540c3e40 2024-08-18T13:49:42.934 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f514a7fc700 1 -- 172.21.15.118:0/182198537 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5373 (secure 0 0 0) 0x7f514c04fa10 con 0x7f51540c3e40 2024-08-18T13:49:42.935 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:42.935 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:38.654010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:42.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5134060860 msgr2=0x7f5134062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:42.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5134060860 0x7f5134062d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f513c009d10 tx=0x7f513c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3e40 msgr2=0x7f5154142920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:42.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3e40 0x7f5154142920 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c510 tx=0x7f514c007f30 comp rx=0 tx=0).stop 2024-08-18T13:49:42.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 shutdown_connections 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f5134060860 0x7f5134062d10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51540c3e40 0x7f5154142920 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51540c3a50 0x7f51541423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 --2- 172.21.15.118:0/182198537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51540c3680 0x7f51540c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 >> 172.21.15.118:0/182198537 conn(0x7f515406c020 msgr2=0x7f51540be230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.929+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 shutdown_connections 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:42.933+0000 7f515359e700 1 -- 172.21.15.118:0/182198537 wait complete. 2024-08-18T13:49:42.938 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 32 2024-08-18T13:49:42.949 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 32, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 32, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5464": { "addr": "172.21.15.178:6835/2054758182", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6832", "nonce": 2054758182, "type": "v2" }, { "addr": "172.21.15.178:6835", "nonce": 2054758182, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5464, "incarnation": 31, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:reconnect", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:49:38.654010+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5464 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:49:44.146 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:49:44.149 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:49:44.949 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:45.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.061+0000 7fefa7995700 1 Processor -- start 2024-08-18T13:49:45.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.061+0000 7fefa7995700 1 -- start start 2024-08-18T13:49:45.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.061+0000 7fefa7995700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0152410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:45.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.061+0000 7fefa7995700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fefa00bd580 0x7fefa006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:45.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.061+0000 7fefa7995700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa0151f80 con 0x7fefa00bea90 2024-08-18T13:49:45.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.061+0000 7fefa7995700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa0151990 con 0x7fefa00bd580 2024-08-18T13:49:45.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.061+0000 7fefa7995700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa014c0a0 con 0x7fefa014bcd0 2024-08-18T13:49:45.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa5f32700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fefa00bea90 0x7fefa006e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:44614/0 (socket says 172.21.15.118:44614) 2024-08-18T13:49:45.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa5f32700 1 -- 172.21.15.118:0/2924740050 learned_addr learned my addr 172.21.15.118:0/2924740050 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:45.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa4f30700 1 --2- 172.21.15.118:0/2924740050 >> v2:172.21.15.178:3300/0 conn(0x7fefa00bd580 0x7fefa006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:45.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa5731700 1 --2- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:45.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fef97fff700 1 -- 172.21.15.118:0/2924740050 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458315558 0 0) 0x7fefa0151f80 con 0x7fefa00bea90 2024-08-18T13:49:45.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fef97fff700 1 -- 172.21.15.118:0/2924740050 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef8c003570 con 0x7fefa00bea90 2024-08-18T13:49:45.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa4f30700 1 -- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 msgr2=0x7fefa0152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:45.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa4f30700 1 --2- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa4f30700 1 -- 172.21.15.118:0/2924740050 >> v1:172.21.15.118:6789/0 conn(0x7fefa00bea90 legacy=0x7fefa006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:45.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fef97fff700 1 -- 172.21.15.118:0/2924740050 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3085169115 0 0) 0x7fef8c003570 con 0x7fefa00bea90 2024-08-18T13:49:45.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa4f30700 1 -- 172.21.15.118:0/2924740050 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0152950 con 0x7fefa00bd580 2024-08-18T13:49:45.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa4f30700 1 --2- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa006e290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef9000aa90 tx=0x7fef9000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ebc4f68ff137998 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:45.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fef97fff700 1 -- 172.21.15.118:0/2924740050 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef980017f0 con 0x7fefa00bd580 2024-08-18T13:49:45.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fef97fff700 1 -- 172.21.15.118:0/2924740050 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9000c6f0 con 0x7fefa00bd580 2024-08-18T13:49:45.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fef97fff700 1 -- 172.21.15.118:0/2924740050 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9000ca70 con 0x7fefa00bd580 2024-08-18T13:49:45.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 -- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 msgr2=0x7fefa006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:45.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 --2- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa006e290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef9000aa90 tx=0x7fef9000a330 comp rx=0 tx=0).stop 2024-08-18T13:49:45.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 -- 172.21.15.118:0/2924740050 shutdown_connections 2024-08-18T13:49:45.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 --2- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa006e290 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 --2- 172.21.15.118:0/2924740050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 -- 172.21.15.118:0/2924740050 >> 172.21.15.118:0/2924740050 conn(0x7fefa00c7c40 msgr2=0x7fefa00c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:45.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 -- 172.21.15.118:0/2924740050 shutdown_connections 2024-08-18T13:49:45.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.065+0000 7fefa7995700 1 -- 172.21.15.118:0/2924740050 wait complete. 2024-08-18T13:49:45.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.069+0000 7fefa7995700 1 Processor -- start 2024-08-18T13:49:45.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.069+0000 7fefa7995700 1 -- start start 2024-08-18T13:49:45.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.069+0000 7fefa7995700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa01656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:45.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefa00bea90 0x7fefa01553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:45.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:45.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0161740 con 0x7fefa00bea90 2024-08-18T13:49:45.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0157d10 con 0x7fefa00bd580 2024-08-18T13:49:45.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0161950 con 0x7fefa014bcd0 2024-08-18T13:49:45.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa4f30700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefa00bea90 0x7fefa01553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:45.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5731700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa01656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:45.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa4f30700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefa00bea90 0x7fefa01553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40358/0 (socket says 172.21.15.118:40358) 2024-08-18T13:49:45.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa4f30700 1 -- 172.21.15.118:0/2659643639 learned_addr learned my addr 172.21.15.118:0/2659643639 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:45.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5f32700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:45.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5731700 1 -- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 msgr2=0x7fefa0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:45.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5731700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5731700 1 -- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefa00bea90 msgr2=0x7fefa01553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:45.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5731700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefa00bea90 0x7fefa01553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5731700 1 -- 172.21.15.118:0/2659643639 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9000a5d0 con 0x7fefa00bd580 2024-08-18T13:49:45.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5f32700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:49:45.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa4f30700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefa00bea90 0x7fefa01553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:49:45.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa5731700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa01656a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00ea50 tx=0x7fef9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:45.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fef957fa700 1 -- 172.21.15.118:0/2659643639 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00ee90 con 0x7fefa00bd580 2024-08-18T13:49:45.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fef957fa700 1 -- 172.21.15.118:0/2659643639 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00c7f0 con 0x7fefa00bd580 2024-08-18T13:49:45.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa0161c30 con 0x7fefa00bd580 2024-08-18T13:49:45.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fef957fa700 1 -- 172.21.15.118:0/2659643639 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00cb90 con 0x7fefa00bd580 2024-08-18T13:49:45.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa0157a10 con 0x7fefa00bd580 2024-08-18T13:49:45.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.073+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa00642f0 con 0x7fefa00bd580 2024-08-18T13:49:45.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.077+0000 7fef957fa700 1 -- 172.21.15.118:0/2659643639 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fef9c005a00 con 0x7fefa00bd580 2024-08-18T13:49:45.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.077+0000 7fef957fa700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef84060860 0x7fef84062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:45.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.077+0000 7fef957fa700 1 -- 172.21.15.118:0/2659643639 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fef9c07d710 con 0x7fefa00bd580 2024-08-18T13:49:45.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.077+0000 7fefa4f30700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef84060860 0x7fef84062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:45.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.077+0000 7fefa4f30700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef84060860 0x7fef84062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefa0165d10 tx=0x7fef90019000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:45.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.089+0000 7fef957fa700 1 -- 172.21.15.118:0/2659643639 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fef9c04d550 con 0x7fefa00bd580 2024-08-18T13:49:45.293 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefa00bd080 con 0x7fefa00bd580 2024-08-18T13:49:45.294 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fef957fa700 1 -- 172.21.15.118:0/2659643639 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fef9c017090 con 0x7fefa00bd580 2024-08-18T13:49:45.295 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:45.295 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:44.684153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:45.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef84060860 msgr2=0x7fef84062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:45.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef84060860 0x7fef84062d10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefa0165d10 tx=0x7fef90019000 comp rx=0 tx=0).stop 2024-08-18T13:49:45.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 msgr2=0x7fefa01656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:45.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa01656a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00ea50 tx=0x7fef9c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.297 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 shutdown_connections 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef84060860 0x7fef84062d10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fefa014bcd0 0x7fefa0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fefa00bea90 0x7fefa01553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 --2- 172.21.15.118:0/2659643639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefa00bd580 0x7fefa01656a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 >> 172.21.15.118:0/2659643639 conn(0x7fefa00c7c40 msgr2=0x7fefa00c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.289+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 shutdown_connections 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:45.293+0000 7fefa7995700 1 -- 172.21.15.118:0/2659643639 wait complete. 2024-08-18T13:49:45.298 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 33 2024-08-18T13:49:45.309 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:47.310 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:47.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 Processor -- start 2024-08-18T13:49:47.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- start start 2024-08-18T13:49:47.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3680 0x7ff2540673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:47.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff2540c3a50 0x7ff25414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:47.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25406cc90 con 0x7ff2540c3e40 2024-08-18T13:49:47.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540c83a0 con 0x7ff2540c3a50 2024-08-18T13:49:47.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff254153090 con 0x7ff2540c3680 2024-08-18T13:49:47.431 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff24bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff2540c3a50 0x7ff25414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff258c55700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7ff2540c3e40 0x7ff25414d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:44618/0 (socket says 172.21.15.118:44618) 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff258c55700 1 -- 172.21.15.118:0/2296726832 learned_addr learned my addr 172.21.15.118:0/2296726832 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff253fff700 1 --2- 172.21.15.118:0/2296726832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3680 0x7ff2540673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff24bfff700 1 -- 172.21.15.118:0/2296726832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3680 msgr2=0x7ff2540673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff24bfff700 1 --2- 172.21.15.118:0/2296726832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3680 0x7ff2540673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff2537fe700 1 -- 172.21.15.118:0/2296726832 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139227673 0 0) 0x7ff25406cc90 con 0x7ff2540c3e40 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff24bfff700 1 -- 172.21.15.118:0/2296726832 >> v1:172.21.15.118:6789/0 conn(0x7ff2540c3e40 legacy=0x7ff25414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:47.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff24bfff700 1 -- 172.21.15.118:0/2296726832 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540c8660 con 0x7ff2540c3a50 2024-08-18T13:49:47.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff24bfff700 1 --2- 172.21.15.118:0/2296726832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3a50 0x7ff25414cd00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0098e0 tx=0x7ff23c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d50a7409ee7b00c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:47.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff2537fe700 1 -- 172.21.15.118:0/2296726832 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00b640 con 0x7ff2540c3a50 2024-08-18T13:49:47.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff2537fe700 1 -- 172.21.15.118:0/2296726832 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00b7a0 con 0x7ff2540c3a50 2024-08-18T13:49:47.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff2537fe700 1 -- 172.21.15.118:0/2296726832 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00bb40 con 0x7ff2540c3a50 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- 172.21.15.118:0/2296726832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3a50 msgr2=0x7ff25414cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 --2- 172.21.15.118:0/2296726832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3a50 0x7ff25414cd00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0098e0 tx=0x7ff23c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- 172.21.15.118:0/2296726832 shutdown_connections 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 --2- 172.21.15.118:0/2296726832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3a50 0x7ff25414cd00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 --2- 172.21.15.118:0/2296726832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3680 0x7ff2540673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- 172.21.15.118:0/2296726832 >> 172.21.15.118:0/2296726832 conn(0x7ff25406c020 msgr2=0x7ff2540c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- 172.21.15.118:0/2296726832 shutdown_connections 2024-08-18T13:49:47.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.425+0000 7ff259456700 1 -- 172.21.15.118:0/2296726832 wait complete. 2024-08-18T13:49:47.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 Processor -- start 2024-08-18T13:49:47.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 -- start start 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3680 0x7ff2540c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3a50 0x7ff254166610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2540c3e40 0x7ff25413ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff254144340 con 0x7ff2540c3680 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff254146c20 con 0x7ff2540c3e40 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2541670b0 con 0x7ff2540c3a50 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3680 0x7ff2540c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:47.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3680 0x7ff2540c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40376/0 (socket says 172.21.15.118:40376) 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 -- 172.21.15.118:0/3534055065 learned_addr learned my addr 172.21.15.118:0/3534055065 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff24bfff700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3a50 0x7ff254166610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff258c55700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2540c3e40 0x7ff25413ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 -- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3a50 msgr2=0x7ff254166610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3a50 0x7ff254166610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 -- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2540c3e40 msgr2=0x7ff25413ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2540c3e40 0x7ff25413ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 -- 172.21.15.118:0/3534055065 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c009420 con 0x7ff2540c3680 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff258c55700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2540c3e40 0x7ff25413ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff253fff700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3680 0x7ff2540c63a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e950 tx=0x7ff2440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff250ff9700 1 -- 172.21.15.118:0/3534055065 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24400c730 con 0x7ff2540c3680 2024-08-18T13:49:47.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff250ff9700 1 -- 172.21.15.118:0/3534055065 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24400c890 con 0x7ff2540c3680 2024-08-18T13:49:47.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2541452c0 con 0x7ff2540c3680 2024-08-18T13:49:47.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.429+0000 7ff250ff9700 1 -- 172.21.15.118:0/3534055065 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24400cc50 con 0x7ff2540c3680 2024-08-18T13:49:47.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.433+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff254143cf0 con 0x7ff2540c3680 2024-08-18T13:49:47.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.433+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2540713a0 con 0x7ff2540c3680 2024-08-18T13:49:47.440 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.433+0000 7ff250ff9700 1 -- 172.21.15.118:0/3534055065 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff244013680 con 0x7ff2540c3680 2024-08-18T13:49:47.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.437+0000 7ff250ff9700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff234060860 0x7ff234062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:47.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.437+0000 7ff250ff9700 1 -- 172.21.15.118:0/3534055065 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7ff24407db20 con 0x7ff2540c3680 2024-08-18T13:49:47.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.437+0000 7ff24bfff700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff234060860 0x7ff234062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:47.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.437+0000 7ff24bfff700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff234060860 0x7ff234062d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0093f0 tx=0x7ff23c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:47.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.441+0000 7ff250ff9700 1 -- 172.21.15.118:0/3534055065 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff24404da80 con 0x7ff2540c3680 2024-08-18T13:49:47.654 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.649+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2541528a0 con 0x7ff2540c3680 2024-08-18T13:49:47.655 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.649+0000 7ff250ff9700 1 -- 172.21.15.118:0/3534055065 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5370 (secure 0 0 0) 0x7ff2440510a0 con 0x7ff2540c3680 2024-08-18T13:49:47.655 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:47.656 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:44.684153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:47.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff234060860 msgr2=0x7ff234062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:47.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff234060860 0x7ff234062d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0093f0 tx=0x7ff23c017000 comp rx=0 tx=0).stop 2024-08-18T13:49:47.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3680 msgr2=0x7ff2540c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:47.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3680 0x7ff2540c63a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e950 tx=0x7ff2440091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 shutdown_connections 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff234060860 0x7ff234062d10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2540c3e40 0x7ff25413ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2540c3a50 0x7ff254166610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 --2- 172.21.15.118:0/3534055065 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2540c3680 0x7ff2540c63a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 >> 172.21.15.118:0/3534055065 conn(0x7ff25406c020 msgr2=0x7ff2540c0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 shutdown_connections 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:47.653+0000 7ff259456700 1 -- 172.21.15.118:0/3534055065 wait complete. 2024-08-18T13:49:47.660 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 33 2024-08-18T13:49:47.673 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:49.674 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:49.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb767fff700 1 Processor -- start 2024-08-18T13:49:49.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb767fff700 1 -- start start 2024-08-18T13:49:49.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb767fff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb7681561d0 0x7fb76815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:49.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb767fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7680c1110 con 0x7fb7681556f0 2024-08-18T13:49:49.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb767fff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7680c1cb0 con 0x7fb7680684e0 2024-08-18T13:49:49.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb767fff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7680c0b90 con 0x7fb7681561d0 2024-08-18T13:49:49.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb7677fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fb7681561d0 0x7fb76815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:49.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb766ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fb7681556f0 0x7fb7680c15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:44628/0 (socket says 172.21.15.118:44628) 2024-08-18T13:49:49.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.785+0000 7fb766ffd700 1 -- 172.21.15.118:0/3086869640 learned_addr learned my addr 172.21.15.118:0/3086869640 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:49.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb765ffb700 1 -- 172.21.15.118:0/3086869640 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043958108 0 0) 0x7fb7680c1cb0 con 0x7fb7680684e0 2024-08-18T13:49:49.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb765ffb700 1 -- 172.21.15.118:0/3086869640 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb754003570 con 0x7fb7680684e0 2024-08-18T13:49:49.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb7677fe700 1 -- 172.21.15.118:0/3086869640 >> v1:172.21.15.178:6789/0 conn(0x7fb7680684e0 legacy=0x7fb768155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:49.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb765ffb700 1 -- 172.21.15.118:0/3086869640 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778432674 0 0) 0x7fb7680c1110 con 0x7fb7681556f0 2024-08-18T13:49:49.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb7677fe700 1 -- 172.21.15.118:0/3086869640 >> v1:172.21.15.118:6789/0 conn(0x7fb7681556f0 legacy=0x7fb7680c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:49.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb7677fe700 1 -- 172.21.15.118:0/3086869640 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7680c8060 con 0x7fb7681561d0 2024-08-18T13:49:49.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb7677fe700 1 --2- 172.21.15.118:0/3086869640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681561d0 0x7fb76815cb00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb7600098e0 tx=0x7fb7600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7f9db9a09ce559 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:49.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb765ffb700 1 -- 172.21.15.118:0/3086869640 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000b640 con 0x7fb7681561d0 2024-08-18T13:49:49.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb765ffb700 1 -- 172.21.15.118:0/3086869640 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76000b7a0 con 0x7fb7681561d0 2024-08-18T13:49:49.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb765ffb700 1 -- 172.21.15.118:0/3086869640 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000bb20 con 0x7fb7681561d0 2024-08-18T13:49:49.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 -- 172.21.15.118:0/3086869640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681561d0 msgr2=0x7fb76815cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:49.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 --2- 172.21.15.118:0/3086869640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681561d0 0x7fb76815cb00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb7600098e0 tx=0x7fb7600091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:49.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 -- 172.21.15.118:0/3086869640 shutdown_connections 2024-08-18T13:49:49.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 --2- 172.21.15.118:0/3086869640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681561d0 0x7fb76815cb00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:49.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 -- 172.21.15.118:0/3086869640 >> 172.21.15.118:0/3086869640 conn(0x7fb76806c020 msgr2=0x7fb7680c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:49.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 -- 172.21.15.118:0/3086869640 shutdown_connections 2024-08-18T13:49:49.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 -- 172.21.15.118:0/3086869640 wait complete. 2024-08-18T13:49:49.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 Processor -- start 2024-08-18T13:49:49.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.789+0000 7fb767fff700 1 -- start start 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7680684e0 0x7fb768168660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681556f0 0x7fb76806df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7681561d0 0x7fb76806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768152f40 con 0x7fb7681556f0 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768168240 con 0x7fb7680684e0 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7681522c0 con 0x7fb7681561d0 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681556f0 0x7fb76806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb766ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7680684e0 0x7fb768168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681556f0 0x7fb76806df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40388/0 (socket says 172.21.15.118:40388) 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 -- 172.21.15.118:0/3520692632 learned_addr learned my addr 172.21.15.118:0/3520692632 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:49.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7677fe700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7681561d0 0x7fb76806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 -- 172.21.15.118:0/3520692632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7681561d0 msgr2=0x7fb76806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7681561d0 0x7fb76806e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 -- 172.21.15.118:0/3520692632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7680684e0 msgr2=0x7fb768168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7680684e0 0x7fb768168660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 -- 172.21.15.118:0/3520692632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb760009420 con 0x7fb7681556f0 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb7667fc700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681556f0 0x7fb76806df80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb75000c910 tx=0x7fb750002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb75effd700 1 -- 172.21.15.118:0/3520692632 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750012630 con 0x7fb7681556f0 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb75effd700 1 -- 172.21.15.118:0/3520692632 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7500127b0 con 0x7fb7681556f0 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb768152c80 con 0x7fb7681556f0 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb75effd700 1 -- 172.21.15.118:0/3520692632 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750012b70 con 0x7fb7681556f0 2024-08-18T13:49:49.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb768153d30 con 0x7fb7681556f0 2024-08-18T13:49:49.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.793+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb768168380 con 0x7fb7681556f0 2024-08-18T13:49:49.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.797+0000 7fb75effd700 1 -- 172.21.15.118:0/3520692632 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb75000a3b0 con 0x7fb7681556f0 2024-08-18T13:49:49.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.797+0000 7fb75effd700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb748060860 0x7fb748062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:49.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.797+0000 7fb75effd700 1 -- 172.21.15.118:0/3520692632 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fb75007d3d0 con 0x7fb7681556f0 2024-08-18T13:49:49.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.797+0000 7fb766ffd700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb748060860 0x7fb748062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:49.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.797+0000 7fb766ffd700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb748060860 0x7fb748062d10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb758007fd0 tx=0x7fb758007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:49.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:49.801+0000 7fb75effd700 1 -- 172.21.15.118:0/3520692632 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb768168380 con 0x7fb7681556f0 2024-08-18T13:49:50.010 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.005+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb768168380 con 0x7fb7681556f0 2024-08-18T13:49:50.011 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.005+0000 7fb75effd700 1 -- 172.21.15.118:0/3520692632 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fb768168380 con 0x7fb7681556f0 2024-08-18T13:49:50.011 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:50.011 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:44.684153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb748060860 msgr2=0x7fb748062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb748060860 0x7fb748062d10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb758007fd0 tx=0x7fb758007d30 comp rx=0 tx=0).stop 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681556f0 msgr2=0x7fb76806df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681556f0 0x7fb76806df80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb75000c910 tx=0x7fb750002700 comp rx=0 tx=0).stop 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 shutdown_connections 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb748060860 0x7fb748062d10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb7681561d0 0x7fb76806e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb7681556f0 0x7fb76806df80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 --2- 172.21.15.118:0/3520692632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7680684e0 0x7fb768168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:50.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 >> 172.21.15.118:0/3520692632 conn(0x7fb76806c020 msgr2=0x7fb7680673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:50.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 shutdown_connections 2024-08-18T13:49:50.015 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:50.009+0000 7fb767fff700 1 -- 172.21.15.118:0/3520692632 wait complete. 2024-08-18T13:49:50.016 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 33 2024-08-18T13:49:50.025 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:52.026 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:52.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6a44700 1 Processor -- start 2024-08-18T13:49:52.143 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6a44700 1 -- start start 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6a44700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2d00684e0 0x7fd2d0155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6a44700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd2d00be4b0 0x7fd2d0156000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6a44700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d00c1110 con 0x7fd2d01556f0 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6a44700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d00c0b90 con 0x7fd2d00684e0 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6a44700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d00c1cb0 con 0x7fd2d00be4b0 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2d00684e0 0x7fd2d0155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd2d00684e0 0x7fd2d0155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40400/0 (socket says 172.21.15.118:40400) 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 -- 172.21.15.118:0/2750917708 learned_addr learned my addr 172.21.15.118:0/2750917708 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d6243700 1 --2- 172.21.15.118:0/2750917708 >> v2:172.21.15.178:3300/0 conn(0x7fd2d00be4b0 0x7fd2d0156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 -- 172.21.15.118:0/2750917708 >> v2:172.21.15.178:3300/0 conn(0x7fd2d00be4b0 msgr2=0x7fd2d0156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:52.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 --2- 172.21.15.118:0/2750917708 >> v2:172.21.15.178:3300/0 conn(0x7fd2d00be4b0 0x7fd2d0156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.146 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 -- 172.21.15.118:0/2750917708 >> v1:172.21.15.118:6789/0 conn(0x7fd2d01556f0 legacy=0x7fd2d00c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:52.146 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 -- 172.21.15.118:0/2750917708 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d00c8060 con 0x7fd2d00684e0 2024-08-18T13:49:52.146 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d5241700 1 --2- 172.21.15.118:0/2750917708 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d00684e0 0x7fd2d0155ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00be270 tx=0x7fd2c400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad826db8e5dc21a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:52.146 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d4a40700 1 -- 172.21.15.118:0/2750917708 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4010070 con 0x7fd2d00684e0 2024-08-18T13:49:52.146 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.137+0000 7fd2d4a40700 1 -- 172.21.15.118:0/2750917708 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c400de90 con 0x7fd2d00684e0 2024-08-18T13:49:52.146 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d4a40700 1 -- 172.21.15.118:0/2750917708 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400b320 con 0x7fd2d00684e0 2024-08-18T13:49:52.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2750917708 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d00684e0 msgr2=0x7fd2d0155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:52.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2750917708 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d00684e0 0x7fd2d0155ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00be270 tx=0x7fd2c400ba50 comp rx=0 tx=0).stop 2024-08-18T13:49:52.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2750917708 shutdown_connections 2024-08-18T13:49:52.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2750917708 >> v2:172.21.15.178:3300/0 conn(0x7fd2d00be4b0 0x7fd2d0156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2750917708 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d00684e0 0x7fd2d0155ac0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2750917708 >> 172.21.15.118:0/2750917708 conn(0x7fd2d006c020 msgr2=0x7fd2d00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:52.147 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2750917708 shutdown_connections 2024-08-18T13:49:52.148 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.141+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2750917708 wait complete. 2024-08-18T13:49:52.149 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 Processor -- start 2024-08-18T13:49:52.149 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 -- start start 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2d00684e0 0x7fd2d01486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2d00be4b0 0x7fd2d0158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d01556f0 0x7fd2d0164480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d01482b0 con 0x7fd2d01556f0 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d0147fd0 con 0x7fd2d00be4b0 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d01476a0 con 0x7fd2d00684e0 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d01556f0 0x7fd2d0164480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d01556f0 0x7fd2d0164480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:40412/0 (socket says 172.21.15.118:40412) 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 -- 172.21.15.118:0/2394660325 learned_addr learned my addr 172.21.15.118:0/2394660325 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d5241700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2d00be4b0 0x7fd2d0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:52.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d5a42700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2d00684e0 0x7fd2d01486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 -- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2d00684e0 msgr2=0x7fd2d01486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2d00684e0 0x7fd2d01486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 -- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2d00be4b0 msgr2=0x7fd2d0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2d00be4b0 0x7fd2d0158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 -- 172.21.15.118:0/2394660325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c400bcf0 con 0x7fd2d01556f0 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6243700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d01556f0 0x7fd2d0164480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800d860 tx=0x7fd2c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2c1ffb700 1 -- 172.21.15.118:0/2394660325 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80125f0 con 0x7fd2d01556f0 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2c1ffb700 1 -- 172.21.15.118:0/2394660325 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8012770 con 0x7fd2d01556f0 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d0147980 con 0x7fd2d01556f0 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2c1ffb700 1 -- 172.21.15.118:0/2394660325 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8012b30 con 0x7fd2d01556f0 2024-08-18T13:49:52.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.145+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d014a740 con 0x7fd2d01556f0 2024-08-18T13:49:52.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.149+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d01641a0 con 0x7fd2d01556f0 2024-08-18T13:49:52.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.149+0000 7fd2c1ffb700 1 -- 172.21.15.118:0/2394660325 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd2c801e460 con 0x7fd2d01556f0 2024-08-18T13:49:52.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.149+0000 7fd2c1ffb700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd2b4060860 0x7fd2b4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:52.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.149+0000 7fd2c1ffb700 1 -- 172.21.15.118:0/2394660325 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fd2c807d670 con 0x7fd2d01556f0 2024-08-18T13:49:52.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.149+0000 7fd2d5a42700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd2b4060860 0x7fd2b4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:52.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.149+0000 7fd2d5a42700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd2b4060860 0x7fd2b4062d10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc000c00 tx=0x7fd2cc007ea0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:52.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.153+0000 7fd2c1ffb700 1 -- 172.21.15.118:0/2394660325 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd2d01641a0 con 0x7fd2d01556f0 2024-08-18T13:49:52.362 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.357+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d01641a0 con 0x7fd2d01556f0 2024-08-18T13:49:52.363 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.357+0000 7fd2c1ffb700 1 -- 172.21.15.118:0/2394660325 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5376 (secure 0 0 0) 0x7fd2d01641a0 con 0x7fd2d01556f0 2024-08-18T13:49:52.363 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:52.363 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:50.714819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:clientreplay","state_seq":35,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:52.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd2b4060860 msgr2=0x7fd2b4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd2b4060860 0x7fd2b4062d10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc000c00 tx=0x7fd2cc007ea0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d01556f0 msgr2=0x7fd2d0164480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d01556f0 0x7fd2d0164480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800d860 tx=0x7fd2c8008040 comp rx=0 tx=0).stop 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 shutdown_connections 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd2b4060860 0x7fd2b4062d10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d01556f0 0x7fd2d0164480 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2d00be4b0 0x7fd2d0158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 --2- 172.21.15.118:0/2394660325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2d00684e0 0x7fd2d01486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 >> 172.21.15.118:0/2394660325 conn(0x7fd2d006c020 msgr2=0x7fd2d006e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:52.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 shutdown_connections 2024-08-18T13:49:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:52.361+0000 7fd2d6a44700 1 -- 172.21.15.118:0/2394660325 wait complete. 2024-08-18T13:49:52.367 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 34 2024-08-18T13:49:52.376 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:49:54.377 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:54.491 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bdaab9700 1 Processor -- start 2024-08-18T13:49:54.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bdaab9700 1 -- start start 2024-08-18T13:49:54.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bdaab9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd40c0d40 0x7f8bd4159350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:54.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bdaab9700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd4159890 con 0x7f8bd414f800 2024-08-18T13:49:54.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bdaab9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd41557f0 con 0x7f8bd414f430 2024-08-18T13:49:54.492 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bdaab9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd4067e00 con 0x7f8bd40c0d40 2024-08-18T13:49:54.493 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bd3fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f8bd414f800 0x7f8bd4087fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:57442/0 (socket says 172.21.15.118:57442) 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.485+0000 7f8bd3fff700 1 -- 172.21.15.118:0/199813860 learned_addr learned my addr 172.21.15.118:0/199813860 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd9056700 1 --2- 172.21.15.118:0/199813860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd40c0d40 0x7f8bd4159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd37fe700 1 -- 172.21.15.118:0/199813860 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203798501 0 0) 0x7f8bbc001490 con 0x7f8bd414f800 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd37fe700 1 -- 172.21.15.118:0/199813860 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bc0003570 con 0x7f8bd414f800 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd37fe700 1 -- 172.21.15.118:0/199813860 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924595383 0 0) 0x7f8bd41557f0 con 0x7f8bd414f430 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd37fe700 1 -- 172.21.15.118:0/199813860 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bbc001490 con 0x7f8bd414f430 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd9056700 1 -- 172.21.15.118:0/199813860 >> v1:172.21.15.178:6789/0 conn(0x7f8bd414f430 legacy=0x7f8bd4155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd9056700 1 -- 172.21.15.118:0/199813860 >> v1:172.21.15.118:6789/0 conn(0x7f8bd414f800 legacy=0x7f8bd4087fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:54.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd9056700 1 -- 172.21.15.118:0/199813860 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40680a0 con 0x7f8bd40c0d40 2024-08-18T13:49:54.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd9056700 1 --2- 172.21.15.118:0/199813860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd40c0d40 0x7f8bd4159350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009910 tx=0x7f8bcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=557d408d4e3e6fb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:54.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd37fe700 1 -- 172.21.15.118:0/199813860 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc011630 con 0x7f8bd40c0d40 2024-08-18T13:49:54.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd37fe700 1 -- 172.21.15.118:0/199813860 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc011790 con 0x7f8bd40c0d40 2024-08-18T13:49:54.495 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bd37fe700 1 -- 172.21.15.118:0/199813860 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc011b30 con 0x7f8bd40c0d40 2024-08-18T13:49:54.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bdaab9700 1 -- 172.21.15.118:0/199813860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd40c0d40 msgr2=0x7f8bd4159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:54.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/199813860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd40c0d40 0x7f8bd4159350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009910 tx=0x7f8bcc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bdaab9700 1 -- 172.21.15.118:0/199813860 shutdown_connections 2024-08-18T13:49:54.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/199813860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd40c0d40 0x7f8bd4159350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bdaab9700 1 -- 172.21.15.118:0/199813860 >> 172.21.15.118:0/199813860 conn(0x7f8bd4068970 msgr2=0x7f8bd4068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:54.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bdaab9700 1 -- 172.21.15.118:0/199813860 shutdown_connections 2024-08-18T13:49:54.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.489+0000 7f8bdaab9700 1 -- 172.21.15.118:0/199813860 wait complete. 2024-08-18T13:49:54.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 Processor -- start 2024-08-18T13:49:54.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 -- start start 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bd40c0d40 0x7f8bd40afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bd414f430 0x7f8bd41660b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd414f800 0x7f8bd40ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40ca1a0 con 0x7f8bd40c0d40 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40cb3c0 con 0x7f8bd414f430 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bdaab9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40cb660 con 0x7f8bd414f800 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bd8855700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bd40c0d40 0x7f8bd40afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bd8855700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bd40c0d40 0x7f8bd40afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33404/0 (socket says 172.21.15.118:33404) 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bd8855700 1 -- 172.21.15.118:0/2810471861 learned_addr learned my addr 172.21.15.118:0/2810471861 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:54.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bd9056700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd414f800 0x7f8bd40ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.493+0000 7f8bd3fff700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bd414f430 0x7f8bd41660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd8855700 1 -- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd414f800 msgr2=0x7f8bd40ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd8855700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd414f800 0x7f8bd40ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd8855700 1 -- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bd414f430 msgr2=0x7f8bd41660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd8855700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bd414f430 0x7f8bd41660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd8855700 1 -- 172.21.15.118:0/2810471861 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009420 con 0x7f8bd40c0d40 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd3fff700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bd414f430 0x7f8bd41660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd8855700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bd40c0d40 0x7f8bd40afbc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400c790 tx=0x7f8bc4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd0ff9700 1 -- 172.21.15.118:0/2810471861 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4010510 con 0x7f8bd40c0d40 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd0ff9700 1 -- 172.21.15.118:0/2810471861 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4010670 con 0x7f8bd40c0d40 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd40cb920 con 0x7f8bd40c0d40 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd0ff9700 1 -- 172.21.15.118:0/2810471861 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4010a30 con 0x7f8bd40c0d40 2024-08-18T13:49:54.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd40c9e30 con 0x7f8bd40c0d40 2024-08-18T13:49:54.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd40642f0 con 0x7f8bd40c0d40 2024-08-18T13:49:54.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd0ff9700 1 -- 172.21.15.118:0/2810471861 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8bc401b3b0 con 0x7f8bd40c0d40 2024-08-18T13:49:54.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd0ff9700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8bb4060890 0x7f8bb4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:54.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd0ff9700 1 -- 172.21.15.118:0/2810471861 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7f8bc407c960 con 0x7f8bd40c0d40 2024-08-18T13:49:54.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd3fff700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8bb4060890 0x7f8bb4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:54.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.497+0000 7f8bd3fff700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8bb4060890 0x7f8bb4062d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4067790 tx=0x7f8bbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:54.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.505+0000 7f8bd0ff9700 1 -- 172.21.15.118:0/2810471861 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8bc404c840 con 0x7f8bd40c0d40 2024-08-18T13:49:54.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.701+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bd405e280 con 0x7f8bd40c0d40 2024-08-18T13:49:54.708 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.701+0000 7f8bd0ff9700 1 -- 172.21.15.118:0/2810471861 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5376 (secure 0 0 0) 0x7f8bc404fe60 con 0x7f8bd40c0d40 2024-08-18T13:49:54.709 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:54.709 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:50.714819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:clientreplay","state_seq":35,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8bb4060890 msgr2=0x7f8bb4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8bb4060890 0x7f8bb4062d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4067790 tx=0x7f8bbc009040 comp rx=0 tx=0).stop 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bd40c0d40 msgr2=0x7f8bd40afbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bd40c0d40 0x7f8bd40afbc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400c790 tx=0x7f8bc4007e10 comp rx=0 tx=0).stop 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 shutdown_connections 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8bb4060890 0x7f8bb4062d40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bd414f800 0x7f8bd40ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.711 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bd414f430 0x7f8bd41660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 --2- 172.21.15.118:0/2810471861 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bd40c0d40 0x7f8bd40afbc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:54.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 >> 172.21.15.118:0/2810471861 conn(0x7f8bd4068970 msgr2=0x7f8bd414ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:54.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 shutdown_connections 2024-08-18T13:49:54.712 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:54.705+0000 7f8bdaab9700 1 -- 172.21.15.118:0/2810471861 wait complete. 2024-08-18T13:49:54.712 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 34 2024-08-18T13:49:54.723 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 34, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 34, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5464": { "addr": "172.21.15.178:6835/2054758182", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6832", "nonce": 2054758182, "type": "v2" }, { "addr": "172.21.15.178:6835", "nonce": 2054758182, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5464, "incarnation": 31, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:clientreplay", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:49:50.714819+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5464 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:49:56.724 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:49:56.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.833+0000 7fef5ffff700 1 Processor -- start 2024-08-18T13:49:56.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.833+0000 7fef5ffff700 1 -- start start 2024-08-18T13:49:56.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c46f0 0x7fef60156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:56.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fef60067d50 0x7fef60153980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:56.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef601534f0 con 0x7fef600c4320 2024-08-18T13:49:56.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef601574c0 con 0x7fef60067d50 2024-08-18T13:49:56.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef60152f70 con 0x7fef600c46f0 2024-08-18T13:49:56.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fef60067d50 0x7fef60153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:56.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fef60067d50 0x7fef60153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33416/0 (socket says 172.21.15.118:33416) 2024-08-18T13:49:56.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5effd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fef600c4320 0x7fef6005e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:57444/0 (socket says 172.21.15.118:57444) 2024-08-18T13:49:56.843 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 -- 172.21.15.118:0/1209359323 learned_addr learned my addr 172.21.15.118:0/1209359323 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5e7fc700 1 --2- 172.21.15.118:0/1209359323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c46f0 0x7fef60156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 -- 172.21.15.118:0/1209359323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c46f0 msgr2=0x7fef60156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 --2- 172.21.15.118:0/1209359323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c46f0 0x7fef60156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 -- 172.21.15.118:0/1209359323 >> v1:172.21.15.118:6789/0 conn(0x7fef600c4320 legacy=0x7fef6005e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 -- 172.21.15.118:0/1209359323 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef60156c20 con 0x7fef60067d50 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5e7fc700 1 --2- 172.21.15.118:0/1209359323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c46f0 0x7fef60156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5f7fe700 1 --2- 172.21.15.118:0/1209359323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef60067d50 0x7fef60153980 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef58009f20 tx=0x7fef58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60f7afd0c6bcc12 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5dffb700 1 -- 172.21.15.118:0/1209359323 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5800e070 con 0x7fef60067d50 2024-08-18T13:49:56.844 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5dffb700 1 -- 172.21.15.118:0/1209359323 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5800b980 con 0x7fef60067d50 2024-08-18T13:49:56.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5dffb700 1 -- 172.21.15.118:0/1209359323 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef580092e0 con 0x7fef60067d50 2024-08-18T13:49:56.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- 172.21.15.118:0/1209359323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef60067d50 msgr2=0x7fef60153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:56.845 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 --2- 172.21.15.118:0/1209359323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef60067d50 0x7fef60153980 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef58009f20 tx=0x7fef58009840 comp rx=0 tx=0).stop 2024-08-18T13:49:56.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- 172.21.15.118:0/1209359323 shutdown_connections 2024-08-18T13:49:56.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 --2- 172.21.15.118:0/1209359323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef60067d50 0x7fef60153980 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:56.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 --2- 172.21.15.118:0/1209359323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c46f0 0x7fef60156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:56.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- 172.21.15.118:0/1209359323 >> 172.21.15.118:0/1209359323 conn(0x7fef600c81e0 msgr2=0x7fef600c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:56.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- 172.21.15.118:0/1209359323 shutdown_connections 2024-08-18T13:49:56.846 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.837+0000 7fef5ffff700 1 -- 172.21.15.118:0/1209359323 wait complete. 2024-08-18T13:49:56.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.841+0000 7fef5ffff700 1 Processor -- start 2024-08-18T13:49:56.847 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.841+0000 7fef5ffff700 1 -- start start 2024-08-18T13:49:56.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.841+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef60067d50 0x7fef601520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:56.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.841+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c4320 0x7fef600c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:56.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.841+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef600c46f0 0x7fef601660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:56.849 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef60156670 con 0x7fef600c46f0 2024-08-18T13:49:56.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5ffff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef60151c50 con 0x7fef60067d50 2024-08-18T13:49:56.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5ffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6014d8b0 con 0x7fef600c4320 2024-08-18T13:49:56.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef600c46f0 0x7fef601660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:56.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef600c46f0 0x7fef601660a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33428/0 (socket says 172.21.15.118:33428) 2024-08-18T13:49:56.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 -- 172.21.15.118:0/766131578 learned_addr learned my addr 172.21.15.118:0/766131578 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:49:56.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5e7fc700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c4320 0x7fef600c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:56.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5effd700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef60067d50 0x7fef601520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:56.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 -- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c4320 msgr2=0x7fef600c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:56.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c4320 0x7fef600c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:56.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 -- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef60067d50 msgr2=0x7fef601520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:56.851 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef60067d50 0x7fef601520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:56.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 -- 172.21.15.118:0/766131578 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58009ae0 con 0x7fef600c46f0 2024-08-18T13:49:56.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5f7fe700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef600c46f0 0x7fef601660a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef58009880 tx=0x7fef58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:56.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef46ffd700 1 -- 172.21.15.118:0/766131578 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef580032f0 con 0x7fef600c46f0 2024-08-18T13:49:56.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef46ffd700 1 -- 172.21.15.118:0/766131578 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef58012af0 con 0x7fef600c46f0 2024-08-18T13:49:56.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef60152c40 con 0x7fef600c46f0 2024-08-18T13:49:56.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef46ffd700 1 -- 172.21.15.118:0/766131578 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58012e90 con 0x7fef600c46f0 2024-08-18T13:49:56.852 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.845+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef60155dd0 con 0x7fef600c46f0 2024-08-18T13:49:56.853 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.849+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef600713a0 con 0x7fef600c46f0 2024-08-18T13:49:56.853 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.849+0000 7fef46ffd700 1 -- 172.21.15.118:0/766131578 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fef5801d630 con 0x7fef600c46f0 2024-08-18T13:49:56.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.849+0000 7fef46ffd700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef4c060860 0x7fef4c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:49:56.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.849+0000 7fef46ffd700 1 -- 172.21.15.118:0/766131578 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fef5807e7e0 con 0x7fef600c46f0 2024-08-18T13:49:56.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.849+0000 7fef5effd700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef4c060860 0x7fef4c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:49:56.855 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.849+0000 7fef5effd700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef4c060860 0x7fef4c062d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef50007180 tx=0x7fef50007f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:49:56.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:56.857+0000 7fef46ffd700 1 -- 172.21.15.118:0/766131578 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fef5804e6c0 con 0x7fef600c46f0 2024-08-18T13:49:57.071 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.065+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef60159850 con 0x7fef600c46f0 2024-08-18T13:49:57.072 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.065+0000 7fef46ffd700 1 -- 172.21.15.118:0/766131578 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fef58051ce0 con 0x7fef600c46f0 2024-08-18T13:49:57.072 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:49:57.072 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:49:57.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef4c060860 msgr2=0x7fef4c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:57.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef4c060860 0x7fef4c062d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef50007180 tx=0x7fef50007f90 comp rx=0 tx=0).stop 2024-08-18T13:49:57.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef600c46f0 msgr2=0x7fef601660a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:49:57.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef600c46f0 0x7fef601660a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef58009880 tx=0x7fef58017000 comp rx=0 tx=0).stop 2024-08-18T13:49:57.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 shutdown_connections 2024-08-18T13:49:57.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fef4c060860 0x7fef4c062d10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:57.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fef600c46f0 0x7fef601660a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:57.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef600c4320 0x7fef600c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:57.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 --2- 172.21.15.118:0/766131578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef60067d50 0x7fef601520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:49:57.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 >> 172.21.15.118:0/766131578 conn(0x7fef600c81e0 msgr2=0x7fef6014af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:49:57.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 shutdown_connections 2024-08-18T13:49:57.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:49:57.069+0000 7fef5ffff700 1 -- 172.21.15.118:0/766131578 wait complete. 2024-08-18T13:49:57.076 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 35 2024-08-18T13:49:57.086 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=5350 2024-08-18T13:49:57.086 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.d 2024-08-18T13:50:02.087 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2024-08-18T13:50:02.087 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-18T13:50:02.087 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-18T13:50:02.089 INFO:tasks.ceph.mds.d:Started 2024-08-18T13:50:02.090 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:02.163 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-08-18T13:50:02.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.197+0000 7ff8b7481700 1 Processor -- start 2024-08-18T13:50:02.204 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.197+0000 7ff8b7481700 1 -- start start 2024-08-18T13:50:02.206 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff8b00c4320 0x7ff8b0156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0067d50 0x7ff8b0153980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b01534f0 con 0x7ff8b00c46f0 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b01574c0 con 0x7ff8b00c4320 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b0152f70 con 0x7ff8b0067d50 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff8b00c4320 0x7ff8b0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff8b00c4320 0x7ff8b0156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33434/0 (socket says 172.21.15.118:33434) 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 -- 172.21.15.118:0/1927058150 learned_addr learned my addr 172.21.15.118:0/1927058150 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b6c80700 1 --2- 172.21.15.118:0/1927058150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0067d50 0x7ff8b0153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 -- 172.21.15.118:0/1927058150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0067d50 msgr2=0x7ff8b0153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 --2- 172.21.15.118:0/1927058150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0067d50 0x7ff8b0153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 -- 172.21.15.118:0/1927058150 >> v1:172.21.15.178:6789/0 conn(0x7ff8b00c46f0 legacy=0x7ff8b005e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:02.207 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 -- 172.21.15.118:0/1927058150 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b0156c20 con 0x7ff8b00c4320 2024-08-18T13:50:02.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b647f700 1 --2- 172.21.15.118:0/1927058150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0156570 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0099b0 tx=0x7ff8ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedfc3f20df63a85 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b547d700 1 -- 172.21.15.118:0/1927058150 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac009ea0 con 0x7ff8b00c4320 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b547d700 1 -- 172.21.15.118:0/1927058150 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac00b860 con 0x7ff8b00c4320 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b547d700 1 -- 172.21.15.118:0/1927058150 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00bc00 con 0x7ff8b00c4320 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- 172.21.15.118:0/1927058150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 msgr2=0x7ff8b0156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1927058150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0156570 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0099b0 tx=0x7ff8ac009250 comp rx=0 tx=0).stop 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- 172.21.15.118:0/1927058150 shutdown_connections 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1927058150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b0067d50 0x7ff8b0153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1927058150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0156570 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- 172.21.15.118:0/1927058150 >> 172.21.15.118:0/1927058150 conn(0x7ff8b00c81e0 msgr2=0x7ff8b00c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- 172.21.15.118:0/1927058150 shutdown_connections 2024-08-18T13:50:02.209 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.201+0000 7ff8b7481700 1 -- 172.21.15.118:0/1927058150 wait complete. 2024-08-18T13:50:02.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 Processor -- start 2024-08-18T13:50:02.210 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 -- start start 2024-08-18T13:50:02.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b0067d50 0x7ff8b0155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:02.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:02.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b00c46f0 0x7ff8b01524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:02.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b015a7f0 con 0x7ff8b00c4320 2024-08-18T13:50:02.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b0159cb0 con 0x7ff8b0067d50 2024-08-18T13:50:02.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b7481700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b006aef0 con 0x7ff8b00c46f0 2024-08-18T13:50:02.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b5c7e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:02.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b5c7e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33444/0 (socket says 172.21.15.118:33444) 2024-08-18T13:50:02.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b5c7e700 1 -- 172.21.15.118:0/1687240835 learned_addr learned my addr 172.21.15.118:0/1687240835 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:02.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b6c80700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b00c46f0 0x7ff8b01524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:02.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b647f700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b0067d50 0x7ff8b0155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:02.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b6c80700 1 -- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b0067d50 msgr2=0x7ff8b0155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:02.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b6c80700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b0067d50 0x7ff8b0155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b6c80700 1 -- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 msgr2=0x7ff8b0151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:02.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b6c80700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b6c80700 1 -- 172.21.15.118:0/1687240835 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac0094f0 con 0x7ff8b00c46f0 2024-08-18T13:50:02.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.205+0000 7ff8b5c7e700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:02.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8b6c80700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b00c46f0 0x7ff8b01524f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009e50 tx=0x7ff8a800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:02.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8a67fc700 1 -- 172.21.15.118:0/1687240835 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800cb50 con 0x7ff8b00c46f0 2024-08-18T13:50:02.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8a67fc700 1 -- 172.21.15.118:0/1687240835 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a800f040 con 0x7ff8b00c46f0 2024-08-18T13:50:02.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b0159f50 con 0x7ff8b00c46f0 2024-08-18T13:50:02.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8a67fc700 1 -- 172.21.15.118:0/1687240835 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8014480 con 0x7ff8b00c46f0 2024-08-18T13:50:02.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b015a510 con 0x7ff8b00c46f0 2024-08-18T13:50:02.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b00713a0 con 0x7ff8b00c46f0 2024-08-18T13:50:02.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.209+0000 7ff8a67fc700 1 -- 172.21.15.118:0/1687240835 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff8a80147f0 con 0x7ff8b00c46f0 2024-08-18T13:50:02.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.213+0000 7ff8a67fc700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff894060860 0x7ff894062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:02.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.213+0000 7ff8a67fc700 1 -- 172.21.15.118:0/1687240835 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7ff8a807e410 con 0x7ff8b00c46f0 2024-08-18T13:50:02.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.213+0000 7ff8b647f700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff894060860 0x7ff894062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:02.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.213+0000 7ff8b647f700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff894060860 0x7ff894062d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0099b0 tx=0x7ff8ac017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:02.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.221+0000 7ff8a67fc700 1 -- 172.21.15.118:0/1687240835 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff8a804e2d0 con 0x7ff8b00c46f0 2024-08-18T13:50:02.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.445+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8b014ab60 con 0x7ff8b00c46f0 2024-08-18T13:50:02.452 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.445+0000 7ff8a67fc700 1 -- 172.21.15.118:0/1687240835 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5370 (secure 0 0 0) 0x7ff8a80518f0 con 0x7ff8b00c46f0 2024-08-18T13:50:02.452 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:02.452 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff894060860 msgr2=0x7ff894062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff894060860 0x7ff894062d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0099b0 tx=0x7ff8ac017000 comp rx=0 tx=0).stop 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b00c46f0 msgr2=0x7ff8b01524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b00c46f0 0x7ff8b01524f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009e50 tx=0x7ff8a800b040 comp rx=0 tx=0).stop 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 shutdown_connections 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff894060860 0x7ff894062d10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8b00c46f0 0x7ff8b01524f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8b00c4320 0x7ff8b0151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 --2- 172.21.15.118:0/1687240835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8b0067d50 0x7ff8b0155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:02.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 >> 172.21.15.118:0/1687240835 conn(0x7ff8b00c81e0 msgr2=0x7ff8b006a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:02.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 shutdown_connections 2024-08-18T13:50:02.456 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:02.449+0000 7ff8b7481700 1 -- 172.21.15.118:0/1687240835 wait complete. 2024-08-18T13:50:02.456 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 35 2024-08-18T13:50:02.466 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-08-18T13:50:02.778 INFO:tasks.ceph.mds.f.smithi178.stderr:2024-08-18T13:50:02.774+0000 7f5ee6994700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T13:50:04.467 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:04.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5f59e700 1 Processor -- start 2024-08-18T13:50:04.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5f59e700 1 -- start start 2024-08-18T13:50:04.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f6006c6a0 0x7f4f60155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:04.584 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5f59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f6006ca70 0x7f4f600c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5f59e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600c1e80 con 0x7f4f600c90e0 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5f59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f601563d0 con 0x7f4f6006ca70 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f600c9690 con 0x7f4f6006c6a0 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5dd9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f6006ca70 0x7f4f600c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5ed9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f4f600c90e0 0x7f4f6014bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:43236/0 (socket says 172.21.15.118:43236) 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5dd9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4f6006ca70 0x7f4f600c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44034/0 (socket says 172.21.15.118:44034) 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5ed9d700 1 -- 172.21.15.118:0/671140117 learned_addr learned my addr 172.21.15.118:0/671140117 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5e59c700 1 --2- 172.21.15.118:0/671140117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f6006c6a0 0x7f4f60155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5d59a700 1 -- 172.21.15.118:0/671140117 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841473884 0 0) 0x7f4f600c1e80 con 0x7f4f600c90e0 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5d59a700 1 -- 172.21.15.118:0/671140117 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f4c003570 con 0x7f4f600c90e0 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5dd9b700 1 -- 172.21.15.118:0/671140117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f6006c6a0 msgr2=0x7f4f60155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5dd9b700 1 --2- 172.21.15.118:0/671140117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f6006c6a0 0x7f4f60155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.585 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5d59a700 1 -- 172.21.15.118:0/671140117 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2432762145 0 0) 0x7f4f4c003570 con 0x7f4f600c90e0 2024-08-18T13:50:04.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5dd9b700 1 -- 172.21.15.118:0/671140117 >> v1:172.21.15.118:6789/0 conn(0x7f4f600c90e0 legacy=0x7f4f6014bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:04.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5dd9b700 1 -- 172.21.15.118:0/671140117 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600c1550 con 0x7f4f6006ca70 2024-08-18T13:50:04.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5e59c700 1 --2- 172.21.15.118:0/671140117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f6006c6a0 0x7f4f60155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:50:04.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5dd9b700 1 --2- 172.21.15.118:0/671140117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6006ca70 0x7f4f600c2310 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000bdf0 tx=0x7f4f5000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dc7f8c9f026a25 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.577+0000 7f4f5d59a700 1 -- 172.21.15.118:0/671140117 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f580017f0 con 0x7f4f6006ca70 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5d59a700 1 -- 172.21.15.118:0/671140117 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5000daf0 con 0x7f4f6006ca70 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5d59a700 1 -- 172.21.15.118:0/671140117 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5000dc80 con 0x7f4f6006ca70 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- 172.21.15.118:0/671140117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6006ca70 msgr2=0x7f4f600c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/671140117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6006ca70 0x7f4f600c2310 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000bdf0 tx=0x7f4f5000b430 comp rx=0 tx=0).stop 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- 172.21.15.118:0/671140117 shutdown_connections 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/671140117 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6006ca70 0x7f4f600c2310 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/671140117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f6006c6a0 0x7f4f60155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- 172.21.15.118:0/671140117 >> 172.21.15.118:0/671140117 conn(0x7f4f6006f4d0 msgr2=0x7f4f600c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:04.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- 172.21.15.118:0/671140117 shutdown_connections 2024-08-18T13:50:04.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- 172.21.15.118:0/671140117 wait complete. 2024-08-18T13:50:04.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 Processor -- start 2024-08-18T13:50:04.588 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- start start 2024-08-18T13:50:04.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6006c6a0 0x7f4f60169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:04.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f600c90e0 0x7f4f6014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:04.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6016cb60 0x7f4f60068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:04.590 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f60169560 con 0x7f4f6016cb60 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6014f160 con 0x7f4f6006c6a0 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.581+0000 7f4f5f59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6016a470 con 0x7f4f600c90e0 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6016cb60 0x7f4f60068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6016cb60 0x7f4f60068780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44050/0 (socket says 172.21.15.118:44050) 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 -- 172.21.15.118:0/2440230954 learned_addr learned my addr 172.21.15.118:0/2440230954 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5e59c700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6006c6a0 0x7f4f60169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5dd9b700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f600c90e0 0x7f4f6014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 -- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f600c90e0 msgr2=0x7f4f6014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f600c90e0 0x7f4f6014fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 -- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6006c6a0 msgr2=0x7f4f60169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6006c6a0 0x7f4f60169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.591 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 -- 172.21.15.118:0/2440230954 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5000b6d0 con 0x7f4f6016cb60 2024-08-18T13:50:04.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5e59c700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6006c6a0 0x7f4f60169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:50:04.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5ed9d700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6016cb60 0x7f4f60068780 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800cc10 tx=0x7f4f58001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:04.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f567fc700 1 -- 172.21.15.118:0/2440230954 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58004dc0 con 0x7f4f6016cb60 2024-08-18T13:50:04.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f567fc700 1 -- 172.21.15.118:0/2440230954 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5800a5f0 con 0x7f4f6016cb60 2024-08-18T13:50:04.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f60147610 con 0x7f4f6016cb60 2024-08-18T13:50:04.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6014e8c0 con 0x7f4f6016cb60 2024-08-18T13:50:04.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f567fc700 1 -- 172.21.15.118:0/2440230954 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5800a9b0 con 0x7f4f6016cb60 2024-08-18T13:50:04.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f567fc700 1 -- 172.21.15.118:0/2440230954 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4f58007150 con 0x7f4f6016cb60 2024-08-18T13:50:04.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.585+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f600713a0 con 0x7f4f6016cb60 2024-08-18T13:50:04.593 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.589+0000 7f4f567fc700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4f40060860 0x7f4f40062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:04.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.589+0000 7f4f567fc700 1 -- 172.21.15.118:0/2440230954 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7f4f5807c8c0 con 0x7f4f6016cb60 2024-08-18T13:50:04.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.589+0000 7f4f5e59c700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4f40060860 0x7f4f40062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:04.594 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.589+0000 7f4f5e59c700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4f40060860 0x7f4f40062d10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f48006f00 tx=0x7f4f48008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:04.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.593+0000 7f4f567fc700 1 -- 172.21.15.118:0/2440230954 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4f5804c700 con 0x7f4f6016cb60 2024-08-18T13:50:04.856 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.849+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f601509d0 con 0x7f4f6016cb60 2024-08-18T13:50:04.857 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f567fc700 1 -- 172.21.15.118:0/2440230954 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f4f58018090 con 0x7f4f6016cb60 2024-08-18T13:50:04.858 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:04.858 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:04.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4f40060860 msgr2=0x7f4f40062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:04.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4f40060860 0x7f4f40062d10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4f48006f00 tx=0x7f4f48008040 comp rx=0 tx=0).stop 2024-08-18T13:50:04.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6016cb60 msgr2=0x7f4f60068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:04.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6016cb60 0x7f4f60068780 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800cc10 tx=0x7f4f58001640 comp rx=0 tx=0).stop 2024-08-18T13:50:04.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 shutdown_connections 2024-08-18T13:50:04.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4f40060860 0x7f4f40062d10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4f6016cb60 0x7f4f60068780 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f600c90e0 0x7f4f6014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 --2- 172.21.15.118:0/2440230954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6006c6a0 0x7f4f60169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:04.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 >> 172.21.15.118:0/2440230954 conn(0x7f4f6006f4d0 msgr2=0x7f4f6014af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:04.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 shutdown_connections 2024-08-18T13:50:04.861 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:04.853+0000 7f4f5f59e700 1 -- 172.21.15.118:0/2440230954 wait complete. 2024-08-18T13:50:04.861 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:04.875 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2024-08-18T13:50:04.876 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 29 secs before thrashing 2024-08-18T13:50:14.194 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:50:14.196 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:50:33.877 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:33.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.980+0000 7fb90c463700 1 Processor -- start 2024-08-18T13:50:33.985 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.980+0000 7fb90c463700 1 -- start start 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb90414bb10 0x7fb9040c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9040c56a0 0x7fb9040c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9040673d0 con 0x7fb90414b740 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb904151830 con 0x7fb9040c56a0 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb904151ab0 con 0x7fb90414bb10 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90a1ff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fb90414b740 0x7fb904151f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:57146/0 (socket says 172.21.15.118:57146) 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90a1ff700 1 -- 172.21.15.118:0/1290336754 learned_addr learned my addr 172.21.15.118:0/1290336754 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:50:33.987 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9099fe700 1 --2- 172.21.15.118:0/1290336754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb90414bb10 0x7fb9040c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90aa00700 1 --2- 172.21.15.118:0/1290336754 >> v2:172.21.15.178:3300/0 conn(0x7fb9040c56a0 0x7fb9040c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9091fd700 1 -- 172.21.15.118:0/1290336754 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951480722 0 0) 0x7fb9040673d0 con 0x7fb90414b740 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9091fd700 1 -- 172.21.15.118:0/1290336754 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8f0003570 con 0x7fb90414b740 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9099fe700 1 -- 172.21.15.118:0/1290336754 >> v2:172.21.15.178:3300/0 conn(0x7fb9040c56a0 msgr2=0x7fb9040c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9099fe700 1 --2- 172.21.15.118:0/1290336754 >> v2:172.21.15.178:3300/0 conn(0x7fb9040c56a0 0x7fb9040c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9099fe700 1 -- 172.21.15.118:0/1290336754 >> v1:172.21.15.118:6789/0 conn(0x7fb90414b740 legacy=0x7fb904151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9099fe700 1 -- 172.21.15.118:0/1290336754 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040c2e80 con 0x7fb90414bb10 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90aa00700 1 --2- 172.21.15.118:0/1290336754 >> v2:172.21.15.178:3300/0 conn(0x7fb9040c56a0 0x7fb9040c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:33.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9099fe700 1 --2- 172.21.15.118:0/1290336754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb90414bb10 0x7fb9040c37c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b9a0 tx=0x7fb8f40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4a6dc76fb19142 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:33.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9091fd700 1 -- 172.21.15.118:0/1290336754 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400c840 con 0x7fb90414bb10 2024-08-18T13:50:33.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9091fd700 1 -- 172.21.15.118:0/1290336754 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f400c9a0 con 0x7fb90414bb10 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb9091fd700 1 -- 172.21.15.118:0/1290336754 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400cd20 con 0x7fb90414bb10 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- 172.21.15.118:0/1290336754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb90414bb10 msgr2=0x7fb9040c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 --2- 172.21.15.118:0/1290336754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb90414bb10 0x7fb9040c37c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b9a0 tx=0x7fb8f40090f0 comp rx=0 tx=0).stop 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- 172.21.15.118:0/1290336754 shutdown_connections 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 --2- 172.21.15.118:0/1290336754 >> v2:172.21.15.178:3300/0 conn(0x7fb9040c56a0 0x7fb9040c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 --2- 172.21.15.118:0/1290336754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb90414bb10 0x7fb9040c37c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- 172.21.15.118:0/1290336754 >> 172.21.15.118:0/1290336754 conn(0x7fb9040c8e70 msgr2=0x7fb9040c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- 172.21.15.118:0/1290336754 shutdown_connections 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.984+0000 7fb90c463700 1 -- 172.21.15.118:0/1290336754 wait complete. 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 Processor -- start 2024-08-18T13:50:33.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 -- start start 2024-08-18T13:50:33.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9040c56a0 0x7fb904161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:33.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb90414b740 0x7fb90406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:33.992 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb90414bb10 0x7fb9041648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040c67e0 con 0x7fb90414b740 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040c62d0 con 0x7fb90414bb10 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9041617c0 con 0x7fb9040c56a0 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb90414b740 0x7fb90406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb90414b740 0x7fb90406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54288/0 (socket says 172.21.15.118:54288) 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90a1ff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9040c56a0 0x7fb904161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 -- 172.21.15.118:0/2652255459 learned_addr learned my addr 172.21.15.118:0/2652255459 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90aa00700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb90414bb10 0x7fb9041648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 -- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9040c56a0 msgr2=0x7fb904161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9040c56a0 0x7fb904161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 -- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb90414bb10 msgr2=0x7fb9041648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb90414bb10 0x7fb9041648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:33.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 -- 172.21.15.118:0/2652255459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc00de30 con 0x7fb90414b740 2024-08-18T13:50:33.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90aa00700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb90414bb10 0x7fb9041648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:33.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb9099fe700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb90414b740 0x7fb90406c5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb904151bf0 tx=0x7fb8f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:33.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb8fa7fc700 1 -- 172.21.15.118:0/2652255459 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f401b050 con 0x7fb90414b740 2024-08-18T13:50:33.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f400b4e0 con 0x7fb90414b740 2024-08-18T13:50:33.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9040c6500 con 0x7fb90414b740 2024-08-18T13:50:33.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb8fa7fc700 1 -- 172.21.15.118:0/2652255459 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f401c040 con 0x7fb90414b740 2024-08-18T13:50:33.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.988+0000 7fb8fa7fc700 1 -- 172.21.15.118:0/2652255459 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4022640 con 0x7fb90414b740 2024-08-18T13:50:33.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.992+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec0052b0 con 0x7fb90414b740 2024-08-18T13:50:33.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.992+0000 7fb8fa7fc700 1 -- 172.21.15.118:0/2652255459 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb8f4022a30 con 0x7fb90414b740 2024-08-18T13:50:33.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.992+0000 7fb8fa7fc700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb8e8060890 0x7fb8e8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:33.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.992+0000 7fb8fa7fc700 1 -- 172.21.15.118:0/2652255459 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fb8f407f550 con 0x7fb90414b740 2024-08-18T13:50:33.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.992+0000 7fb90a1ff700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb8e8060890 0x7fb8e8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:33.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.992+0000 7fb90a1ff700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb8e8060890 0x7fb8e8062d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb900007fd0 tx=0x7fb900009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:34.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:33.996+0000 7fb8fa7fc700 1 -- 172.21.15.118:0/2652255459 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb8f404f430 con 0x7fb90414b740 2024-08-18T13:50:34.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.236+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8ec005560 con 0x7fb90414b740 2024-08-18T13:50:34.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.236+0000 7fb8fa7fc700 1 -- 172.21.15.118:0/2652255459 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fb8f4052a50 con 0x7fb90414b740 2024-08-18T13:50:34.239 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:34.239 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:34.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.236+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb8e8060890 msgr2=0x7fb8e8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.236+0000 7fb90c463700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb8e8060890 0x7fb8e8062d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb900007fd0 tx=0x7fb900009040 comp rx=0 tx=0).stop 2024-08-18T13:50:34.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.236+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb90414b740 msgr2=0x7fb90406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.236+0000 7fb90c463700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb90414b740 0x7fb90406c5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb904151bf0 tx=0x7fb8f4017000 comp rx=0 tx=0).stop 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 shutdown_connections 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fb8e8060890 0x7fb8e8062d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb90414bb10 0x7fb9041648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb90414b740 0x7fb90406c5f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 --2- 172.21.15.118:0/2652255459 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9040c56a0 0x7fb904161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 >> 172.21.15.118:0/2652255459 conn(0x7fb9040c8e70 msgr2=0x7fb904067580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 shutdown_connections 2024-08-18T13:50:34.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.240+0000 7fb90c463700 1 -- 172.21.15.118:0/2652255459 wait complete. 2024-08-18T13:50:34.243 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:34.253 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2024-08-18T13:50:34.253 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-08-18T13:50:34.253 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:50:34.271 INFO:tasks.ceph.mds.f:Stopped 2024-08-18T13:50:34.271 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:34.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.372+0000 7f6e4c763700 1 Processor -- start 2024-08-18T13:50:34.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.372+0000 7f6e4c763700 1 -- start start 2024-08-18T13:50:34.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.372+0000 7f6e4c763700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4406b620 0x7f6e4406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:34.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.372+0000 7f6e4c763700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6e440c4030 0x7f6e44152c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:34.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.372+0000 7f6e4c763700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4406bf30 con 0x7f6e440c2630 2024-08-18T13:50:34.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.372+0000 7f6e4c763700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44068ac0 con 0x7f6e440c4030 2024-08-18T13:50:34.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.372+0000 7f6e4c763700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44153160 con 0x7f6e4406b620 2024-08-18T13:50:34.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e49cfe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6e440c4030 0x7f6e44152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4ad00700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f6e440c2630 0x7f6e4414c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:58320/0 (socket says 172.21.15.118:58320) 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4a4ff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4406b620 0x7f6e4406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4ad00700 1 -- 172.21.15.118:0/3298524047 learned_addr learned my addr 172.21.15.118:0/3298524047 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e494fd700 1 -- 172.21.15.118:0/3298524047 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608435285 0 0) 0x7f6e4406bf30 con 0x7f6e440c2630 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e494fd700 1 -- 172.21.15.118:0/3298524047 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e30003570 con 0x7f6e440c2630 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4a4ff700 1 -- 172.21.15.118:0/3298524047 >> v2:172.21.15.178:3300/0 conn(0x7f6e440c4030 msgr2=0x7f6e44152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4a4ff700 1 --2- 172.21.15.118:0/3298524047 >> v2:172.21.15.178:3300/0 conn(0x7f6e440c4030 0x7f6e44152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4a4ff700 1 -- 172.21.15.118:0/3298524047 >> v1:172.21.15.178:6789/0 conn(0x7f6e440c2630 legacy=0x7f6e4414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4a4ff700 1 -- 172.21.15.118:0/3298524047 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44068580 con 0x7f6e4406b620 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e49cfe700 1 --2- 172.21.15.118:0/3298524047 >> v2:172.21.15.178:3300/0 conn(0x7f6e440c4030 0x7f6e44152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4a4ff700 1 --2- 172.21.15.118:0/3298524047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4406b620 0x7f6e4406b9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098e0 tx=0x7f6e340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155885e5f81bbe08 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e494fd700 1 -- 172.21.15.118:0/3298524047 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34011630 con 0x7f6e4406b620 2024-08-18T13:50:34.380 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e494fd700 1 -- 172.21.15.118:0/3298524047 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e34011790 con 0x7f6e4406b620 2024-08-18T13:50:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e494fd700 1 -- 172.21.15.118:0/3298524047 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34011b30 con 0x7f6e4406b620 2024-08-18T13:50:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 -- 172.21.15.118:0/3298524047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4406b620 msgr2=0x7f6e4406b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.381 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 --2- 172.21.15.118:0/3298524047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4406b620 0x7f6e4406b9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098e0 tx=0x7f6e340091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 -- 172.21.15.118:0/3298524047 shutdown_connections 2024-08-18T13:50:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 --2- 172.21.15.118:0/3298524047 >> v2:172.21.15.178:3300/0 conn(0x7f6e440c4030 0x7f6e44152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 --2- 172.21.15.118:0/3298524047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e4406b620 0x7f6e4406b9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 -- 172.21.15.118:0/3298524047 >> 172.21.15.118:0/3298524047 conn(0x7f6e440c7c40 msgr2=0x7f6e440c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 -- 172.21.15.118:0/3298524047 shutdown_connections 2024-08-18T13:50:34.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.376+0000 7f6e4c763700 1 -- 172.21.15.118:0/3298524047 wait complete. 2024-08-18T13:50:34.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 Processor -- start 2024-08-18T13:50:34.385 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 -- start start 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4406b620 0x7f6e44162bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440c2630 0x7f6e44151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e440c4030 0x7f6e440bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44163130 con 0x7f6e440c4030 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4406c450 con 0x7f6e4406b620 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4c763700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4414d0e0 con 0x7f6e440c2630 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e440c4030 0x7f6e440bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e440c4030 0x7f6e440bfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54296/0 (socket says 172.21.15.118:54296) 2024-08-18T13:50:34.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 -- 172.21.15.118:0/26333882 learned_addr learned my addr 172.21.15.118:0/26333882 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4a4ff700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4406b620 0x7f6e44162bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e49cfe700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440c2630 0x7f6e44151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 -- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440c2630 msgr2=0x7f6e44151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440c2630 0x7f6e44151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 -- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4406b620 msgr2=0x7f6e44162bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4406b620 0x7f6e44162bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 -- 172.21.15.118:0/26333882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40009420 con 0x7f6e440c4030 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4a4ff700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4406b620 0x7f6e44162bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e4ad00700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e440c4030 0x7f6e440bfef0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c001460 tx=0x7f6e3c001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.380+0000 7f6e3a7fc700 1 -- 172.21.15.118:0/26333882 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011470 con 0x7f6e440c4030 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.384+0000 7f6e3a7fc700 1 -- 172.21.15.118:0/26333882 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c0115d0 con 0x7f6e440c4030 2024-08-18T13:50:34.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.384+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e34009420 con 0x7f6e440c4030 2024-08-18T13:50:34.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.384+0000 7f6e3a7fc700 1 -- 172.21.15.118:0/26333882 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011950 con 0x7f6e440c4030 2024-08-18T13:50:34.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.384+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4414d660 con 0x7f6e440c4030 2024-08-18T13:50:34.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.384+0000 7f6e3a7fc700 1 -- 172.21.15.118:0/26333882 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6e3c011d40 con 0x7f6e440c4030 2024-08-18T13:50:34.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.384+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e440642f0 con 0x7f6e440c4030 2024-08-18T13:50:34.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.384+0000 7f6e3a7fc700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6e28060860 0x7f6e28062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:34.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.388+0000 7f6e4a4ff700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6e28060860 0x7f6e28062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:34.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.388+0000 7f6e3a7fc700 1 -- 172.21.15.118:0/26333882 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7f6e3c07fe40 con 0x7f6e440c4030 2024-08-18T13:50:34.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.388+0000 7f6e4a4ff700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6e28060860 0x7f6e28062d10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e340091e0 tx=0x7f6e34017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:34.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.392+0000 7f6e3a7fc700 1 -- 172.21.15.118:0/26333882 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6e3c04fd10 con 0x7f6e440c4030 2024-08-18T13:50:34.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.588+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e4414ee00 con 0x7f6e440c4030 2024-08-18T13:50:34.592 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.588+0000 7f6e3a7fc700 1 -- 172.21.15.118:0/26333882 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f6e3c015090 con 0x7f6e440c4030 2024-08-18T13:50:34.593 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:34.593 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:34.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6e28060860 msgr2=0x7f6e28062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6e28060860 0x7f6e28062d10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e340091e0 tx=0x7f6e34017000 comp rx=0 tx=0).stop 2024-08-18T13:50:34.595 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e440c4030 msgr2=0x7f6e440bfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e440c4030 0x7f6e440bfef0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c001460 tx=0x7f6e3c001b70 comp rx=0 tx=0).stop 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 shutdown_connections 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6e28060860 0x7f6e28062d10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e440c4030 0x7f6e440bfef0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e440c2630 0x7f6e44151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 --2- 172.21.15.118:0/26333882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4406b620 0x7f6e44162bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 >> 172.21.15.118:0/26333882 conn(0x7f6e440c7c40 msgr2=0x7f6e440c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 shutdown_connections 2024-08-18T13:50:34.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:34.592+0000 7f6e4c763700 1 -- 172.21.15.118:0/26333882 wait complete. 2024-08-18T13:50:34.597 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:34.611 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:50:36.611 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:36.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd80ae7b700 1 Processor -- start 2024-08-18T13:50:36.728 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd80ae7b700 1 -- start start 2024-08-18T13:50:36.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd80ae7b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd8040c3680 0x7fd8040673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:36.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd80ae7b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3a50 0x7fd80414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:36.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd80ae7b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80406cc90 con 0x7fd8040c3e40 2024-08-18T13:50:36.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd80ae7b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8040c83a0 con 0x7fd8040c3680 2024-08-18T13:50:36.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd80ae7b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd804153090 con 0x7fd8040c3a50 2024-08-18T13:50:36.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd809e79700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd8040c3680 0x7fd8040673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:36.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd809678700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3a50 0x7fd80414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:36.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd809e79700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd8040c3680 0x7fd8040673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54310/0 (socket says 172.21.15.118:54310) 2024-08-18T13:50:36.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.724+0000 7fd809e79700 1 -- 172.21.15.118:0/1614693588 learned_addr learned my addr 172.21.15.118:0/1614693588 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:36.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd808e77700 1 -- 172.21.15.118:0/1614693588 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929911694 0 0) 0x7fd80406cc90 con 0x7fd8040c3e40 2024-08-18T13:50:36.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd808e77700 1 -- 172.21.15.118:0/1614693588 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f0003570 con 0x7fd8040c3e40 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd809678700 1 -- 172.21.15.118:0/1614693588 >> v2:172.21.15.118:3300/0 conn(0x7fd8040c3680 msgr2=0x7fd8040673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd809678700 1 --2- 172.21.15.118:0/1614693588 >> v2:172.21.15.118:3300/0 conn(0x7fd8040c3680 0x7fd8040673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd809678700 1 -- 172.21.15.118:0/1614693588 >> v1:172.21.15.178:6789/0 conn(0x7fd8040c3e40 legacy=0x7fd80414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd809678700 1 -- 172.21.15.118:0/1614693588 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040c8660 con 0x7fd8040c3a50 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd809e79700 1 --2- 172.21.15.118:0/1614693588 >> v2:172.21.15.118:3300/0 conn(0x7fd8040c3680 0x7fd8040673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd809678700 1 --2- 172.21.15.118:0/1614693588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3a50 0x7fd80414cd00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd800009e60 tx=0x7fd800009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31b343d119b0521 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd808e77700 1 -- 172.21.15.118:0/1614693588 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80000e070 con 0x7fd8040c3a50 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd808e77700 1 -- 172.21.15.118:0/1614693588 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80000bab0 con 0x7fd8040c3a50 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd808e77700 1 -- 172.21.15.118:0/1614693588 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80000be30 con 0x7fd8040c3a50 2024-08-18T13:50:36.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 -- 172.21.15.118:0/1614693588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3a50 msgr2=0x7fd80414cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:36.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/1614693588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3a50 0x7fd80414cd00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd800009e60 tx=0x7fd800009780 comp rx=0 tx=0).stop 2024-08-18T13:50:36.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 -- 172.21.15.118:0/1614693588 shutdown_connections 2024-08-18T13:50:36.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/1614693588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3a50 0x7fd80414cd00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/1614693588 >> v2:172.21.15.118:3300/0 conn(0x7fd8040c3680 0x7fd8040673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 -- 172.21.15.118:0/1614693588 >> 172.21.15.118:0/1614693588 conn(0x7fd80406c020 msgr2=0x7fd8040c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:36.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 -- 172.21.15.118:0/1614693588 shutdown_connections 2024-08-18T13:50:36.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.728+0000 7fd80ae7b700 1 -- 172.21.15.118:0/1614693588 wait complete. 2024-08-18T13:50:36.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 Processor -- start 2024-08-18T13:50:36.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 -- start start 2024-08-18T13:50:36.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8040c3680 0x7fd80414dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8040c3a50 0x7fd8041636c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3e40 0x7fd80414c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd804142d00 con 0x7fd8040c3a50 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80414cbc0 con 0x7fd8040c3680 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd804162cb0 con 0x7fd8040c3e40 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8040c3a50 0x7fd8041636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8040c3a50 0x7fd8041636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54324/0 (socket says 172.21.15.118:54324) 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 -- 172.21.15.118:0/587041080 learned_addr learned my addr 172.21.15.118:0/587041080 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809e79700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8040c3680 0x7fd80414dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80a67a700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3e40 0x7fd80414c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 -- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3e40 msgr2=0x7fd80414c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3e40 0x7fd80414c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 -- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8040c3680 msgr2=0x7fd80414dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:36.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8040c3680 0x7fd80414dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 -- 172.21.15.118:0/587041080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40094f0 con 0x7fd8040c3a50 2024-08-18T13:50:36.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd809678700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8040c3a50 0x7fd8041636c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd8041531d0 tx=0x7fd800019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:36.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd7f9ffb700 1 -- 172.21.15.118:0/587041080 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800001010 con 0x7fd8040c3a50 2024-08-18T13:50:36.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd7f9ffb700 1 -- 172.21.15.118:0/587041080 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd800021070 con 0x7fd8040c3a50 2024-08-18T13:50:36.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd7f9ffb700 1 -- 172.21.15.118:0/587041080 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800012dc0 con 0x7fd8040c3a50 2024-08-18T13:50:36.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd800009a20 con 0x7fd8040c3a50 2024-08-18T13:50:36.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.732+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80414cdd0 con 0x7fd8040c3a50 2024-08-18T13:50:36.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.736+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8040713a0 con 0x7fd8040c3a50 2024-08-18T13:50:36.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.736+0000 7fd7f9ffb700 1 -- 172.21.15.118:0/587041080 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd800023030 con 0x7fd8040c3a50 2024-08-18T13:50:36.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.736+0000 7fd7f9ffb700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd7e8060890 0x7fd7e8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:36.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.736+0000 7fd7f9ffb700 1 -- 172.21.15.118:0/587041080 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fd80007eeb0 con 0x7fd8040c3a50 2024-08-18T13:50:36.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.736+0000 7fd809e79700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd7e8060890 0x7fd7e8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:36.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.736+0000 7fd809e79700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd7e8060890 0x7fd7e8062d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006d20 tx=0x7fd7f4009250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:36.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.744+0000 7fd7f9ffb700 1 -- 172.21.15.118:0/587041080 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd80004ed90 con 0x7fd8040c3a50 2024-08-18T13:50:36.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8040688c0 con 0x7fd8040c3a50 2024-08-18T13:50:36.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd7f9ffb700 1 -- 172.21.15.118:0/587041080 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fd8000523b0 con 0x7fd8040c3a50 2024-08-18T13:50:36.950 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:36.951 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd7e8060890 msgr2=0x7fd7e8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd7e8060890 0x7fd7e8062d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006d20 tx=0x7fd7f4009250 comp rx=0 tx=0).stop 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8040c3a50 msgr2=0x7fd8041636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8040c3a50 0x7fd8041636c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd8041531d0 tx=0x7fd800019000 comp rx=0 tx=0).stop 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 shutdown_connections 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd7e8060890 0x7fd7e8062d40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8040c3e40 0x7fd80414c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd8040c3a50 0x7fd8041636c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 --2- 172.21.15.118:0/587041080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8040c3680 0x7fd80414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 >> 172.21.15.118:0/587041080 conn(0x7fd80406c020 msgr2=0x7fd804150f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:36.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 shutdown_connections 2024-08-18T13:50:36.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:36.948+0000 7fd80ae7b700 1 -- 172.21.15.118:0/587041080 wait complete. 2024-08-18T13:50:36.954 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:36.965 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:50:38.965 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:39.081 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fb59e700 1 Processor -- start 2024-08-18T13:50:39.082 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fb59e700 1 -- start start 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fb59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc4fc14b840 0x7fc4fc155580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fb59e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc4fc06b180 0x7fc4fc0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fb59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc0c2de0 con 0x7fc4fc069fc0 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fb59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc0c1d80 con 0x7fc4fc14b840 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fb59e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc0c30c0 con 0x7fc4fc06b180 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fa59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc4fc14b840 0x7fc4fc155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fa59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc4fc14b840 0x7fc4fc155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54334/0 (socket says 172.21.15.118:54334) 2024-08-18T13:50:39.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fa59c700 1 -- 172.21.15.118:0/3903617886 learned_addr learned my addr 172.21.15.118:0/3903617886 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:39.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fad9d700 1 --2- 172.21.15.118:0/3903617886 >> v2:172.21.15.178:3300/0 conn(0x7fc4fc06b180 0x7fc4fc0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4f959a700 1 -- 172.21.15.118:0/3903617886 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335736468 0 0) 0x7fc4fc0c2de0 con 0x7fc4fc069fc0 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4f959a700 1 -- 172.21.15.118:0/3903617886 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e4003570 con 0x7fc4fc069fc0 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fa59c700 1 -- 172.21.15.118:0/3903617886 >> v2:172.21.15.178:3300/0 conn(0x7fc4fc06b180 msgr2=0x7fc4fc0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fa59c700 1 --2- 172.21.15.118:0/3903617886 >> v2:172.21.15.178:3300/0 conn(0x7fc4fc06b180 0x7fc4fc0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fa59c700 1 -- 172.21.15.118:0/3903617886 >> v1:172.21.15.178:6789/0 conn(0x7fc4fc069fc0 legacy=0x7fc4fc06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fa59c700 1 -- 172.21.15.118:0/3903617886 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc06a490 con 0x7fc4fc14b840 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.080+0000 7fc4fad9d700 1 --2- 172.21.15.118:0/3903617886 >> v2:172.21.15.178:3300/0 conn(0x7fc4fc06b180 0x7fc4fc0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:39.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fa59c700 1 --2- 172.21.15.118:0/3903617886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc155580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0c0240 tx=0x7fc4ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836b885e03dfd785 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:39.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4f959a700 1 -- 172.21.15.118:0/3903617886 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00b640 con 0x7fc4fc14b840 2024-08-18T13:50:39.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4f959a700 1 -- 172.21.15.118:0/3903617886 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec00b7a0 con 0x7fc4fc14b840 2024-08-18T13:50:39.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4f959a700 1 -- 172.21.15.118:0/3903617886 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00bb60 con 0x7fc4fc14b840 2024-08-18T13:50:39.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 -- 172.21.15.118:0/3903617886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 msgr2=0x7fc4fc155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/3903617886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc155580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0c0240 tx=0x7fc4ec009250 comp rx=0 tx=0).stop 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 -- 172.21.15.118:0/3903617886 shutdown_connections 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/3903617886 >> v2:172.21.15.178:3300/0 conn(0x7fc4fc06b180 0x7fc4fc0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/3903617886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc155580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 -- 172.21.15.118:0/3903617886 >> 172.21.15.118:0/3903617886 conn(0x7fc4fc0c81e0 msgr2=0x7fc4fc0c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 -- 172.21.15.118:0/3903617886 shutdown_connections 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 -- 172.21.15.118:0/3903617886 wait complete. 2024-08-18T13:50:39.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 Processor -- start 2024-08-18T13:50:39.089 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.084+0000 7fc4fb59e700 1 -- start start 2024-08-18T13:50:39.091 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4fc069fc0 0x7fc4fc151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:39.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4fc06b180 0x7fc4fc1653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc1658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc164c90 con 0x7fc4fc14b840 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc067e60 con 0x7fc4fc069fc0 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc164f10 con 0x7fc4fc06b180 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc1658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc1658e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54346/0 (socket says 172.21.15.118:54346) 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 -- 172.21.15.118:0/403914267 learned_addr learned my addr 172.21.15.118:0/403914267 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4f9d9b700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4fc06b180 0x7fc4fc1653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fa59c700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4fc069fc0 0x7fc4fc151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 -- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4fc06b180 msgr2=0x7fc4fc1653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4fc06b180 0x7fc4fc1653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 -- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4fc069fc0 msgr2=0x7fc4fc151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4fc069fc0 0x7fc4fc151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 -- 172.21.15.118:0/403914267 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f4009a20 con 0x7fc4fc14b840 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fad9d700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc1658e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c0f0 tx=0x7fc4f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4ea7fc700 1 -- 172.21.15.118:0/403914267 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4009d20 con 0x7fc4fc14b840 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4ea7fc700 1 -- 172.21.15.118:0/403914267 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f400f040 con 0x7fc4fc14b840 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec0094f0 con 0x7fc4fc14b840 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4ea7fc700 1 -- 172.21.15.118:0/403914267 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f400ce80 con 0x7fc4fc14b840 2024-08-18T13:50:39.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc146280 con 0x7fc4fc14b840 2024-08-18T13:50:39.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.088+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc0713a0 con 0x7fc4fc14b840 2024-08-18T13:50:39.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.092+0000 7fc4ea7fc700 1 -- 172.21.15.118:0/403914267 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc4f4003020 con 0x7fc4fc14b840 2024-08-18T13:50:39.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.092+0000 7fc4ea7fc700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc4dc060890 0x7fc4dc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:39.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.092+0000 7fc4fa59c700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc4dc060890 0x7fc4dc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:39.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.092+0000 7fc4ea7fc700 1 -- 172.21.15.118:0/403914267 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fc4f407d900 con 0x7fc4fc14b840 2024-08-18T13:50:39.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.092+0000 7fc4fa59c700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc4dc060890 0x7fc4dc062d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0c0240 tx=0x7fc4ec017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:39.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.100+0000 7fc4ea7fc700 1 -- 172.21.15.118:0/403914267 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc4f4051270 con 0x7fc4fc14b840 2024-08-18T13:50:39.303 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.300+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4fc165050 con 0x7fc4fc14b840 2024-08-18T13:50:39.304 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.300+0000 7fc4ea7fc700 1 -- 172.21.15.118:0/403914267 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fc4f4050e00 con 0x7fc4fc14b840 2024-08-18T13:50:39.304 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:39.304 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:39.306 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc4dc060890 msgr2=0x7fc4dc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc4dc060890 0x7fc4dc062d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0c0240 tx=0x7fc4ec017000 comp rx=0 tx=0).stop 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 msgr2=0x7fc4fc1658e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc1658e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c0f0 tx=0x7fc4f400b040 comp rx=0 tx=0).stop 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 shutdown_connections 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc4dc060890 0x7fc4dc062d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4fc14b840 0x7fc4fc1658e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4fc06b180 0x7fc4fc1653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 --2- 172.21.15.118:0/403914267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4fc069fc0 0x7fc4fc151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 >> 172.21.15.118:0/403914267 conn(0x7fc4fc0c81e0 msgr2=0x7fc4fc0c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 shutdown_connections 2024-08-18T13:50:39.307 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:39.304+0000 7fc4fb59e700 1 -- 172.21.15.118:0/403914267 wait complete. 2024-08-18T13:50:39.308 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:39.318 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:50:41.319 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:41.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.432+0000 7fda3085c700 1 Processor -- start 2024-08-18T13:50:41.437 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.432+0000 7fda3085c700 1 -- start start 2024-08-18T13:50:41.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda3085c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda2c0ae970 0x7fda2c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda3085c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c06f340 con 0x7fda2c0b15f0 2024-08-18T13:50:41.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda3085c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c0aeeb0 con 0x7fda2c0add40 2024-08-18T13:50:41.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda3085c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c155110 con 0x7fda2c0ae970 2024-08-18T13:50:41.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fda2c0b15f0 0x7fda2c0c7500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:57162/0 (socket says 172.21.15.118:57162) 2024-08-18T13:50:41.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 -- 172.21.15.118:0/1486647721 learned_addr learned my addr 172.21.15.118:0/1486647721 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:50:41.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2bfff700 1 --2- 172.21.15.118:0/1486647721 >> v2:172.21.15.178:3300/0 conn(0x7fda2c0ae970 0x7fda2c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126639079 0 0) 0x7fda2c06f340 con 0x7fda2c0b15f0 2024-08-18T13:50:41.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda0c003570 con 0x7fda2c0b15f0 2024-08-18T13:50:41.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634523856 0 0) 0x7fda2c0aeeb0 con 0x7fda2c0add40 2024-08-18T13:50:41.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda2c06f340 con 0x7fda2c0add40 2024-08-18T13:50:41.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4232655691 0 0) 0x7fda0c003570 con 0x7fda2c0b15f0 2024-08-18T13:50:41.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 >> v2:172.21.15.178:3300/0 conn(0x7fda2c0ae970 msgr2=0x7fda2c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 --2- 172.21.15.118:0/1486647721 >> v2:172.21.15.178:3300/0 conn(0x7fda2c0ae970 0x7fda2c160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 >> v1:172.21.15.178:6789/0 conn(0x7fda2c0add40 legacy=0x7fda2c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c0af150 con 0x7fda2c0b15f0 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fda14001470 con 0x7fda2c0b15f0 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 >> v1:172.21.15.118:6789/0 conn(0x7fda2c0b15f0 legacy=0x7fda2c0c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c003bf0 0x7fda0c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0c004810 0x7fda0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda0c0051c0 0x7fda0c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c155110 con 0x7fda0c004810 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c00c0b0 con 0x7fda0c0051c0 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c00c270 con 0x7fda0c003bf0 2024-08-18T13:50:41.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0c004810 0x7fda0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2bfff700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda0c0051c0 0x7fda0c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2b7fe700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c003bf0 0x7fda0c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 -- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c003bf0 msgr2=0x7fda0c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c003bf0 0x7fda0c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 -- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda0c0051c0 msgr2=0x7fda0c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda0c0051c0 0x7fda0c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda2bfff700 1 -- 172.21.15.118:0/1486647721 reap_dead start 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 -- 172.21.15.118:0/1486647721 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c003570 con 0x7fda0c004810 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.436+0000 7fda23fff700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0c004810 0x7fda0c004c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c2d0 tx=0x7fda2c0b0b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d72d8ccc8464f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1400cba0 con 0x7fda0c004810 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1400cd20 con 0x7fda0c004810 2024-08-18T13:50:41.443 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda2affd700 1 -- 172.21.15.118:0/1486647721 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14005b70 con 0x7fda0c004810 2024-08-18T13:50:41.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- 172.21.15.118:0/1486647721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0c004810 msgr2=0x7fda0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:41.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0c004810 0x7fda0c004c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c2d0 tx=0x7fda2c0b0b10 comp rx=0 tx=0).stop 2024-08-18T13:50:41.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- 172.21.15.118:0/1486647721 shutdown_connections 2024-08-18T13:50:41.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 --2- 172.21.15.118:0/1486647721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0c004810 0x7fda0c004c80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- 172.21.15.118:0/1486647721 >> 172.21.15.118:0/1486647721 conn(0x7fda2c0cc600 msgr2=0x7fda2c0c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- 172.21.15.118:0/1486647721 shutdown_connections 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- 172.21.15.118:0/1486647721 wait complete. 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 Processor -- start 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- start start 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c004810 0x7fda2c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda2c0b0ad0 0x7fda2c0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda2c0c4b10 0x7fda2c167d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c152940 con 0x7fda2c0c4b10 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0c3e10 con 0x7fda2c0b0ad0 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda3085c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0c5170 con 0x7fda0c004810 2024-08-18T13:50:41.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda2c0c4b10 0x7fda2c167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda2c0c4b10 0x7fda2c167d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54364/0 (socket says 172.21.15.118:54364) 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda2bfff700 1 -- 172.21.15.118:0/851033223 learned_addr learned my addr 172.21.15.118:0/851033223 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda23fff700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda2c0b0ad0 0x7fda2c0c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.440+0000 7fda2b7fe700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c004810 0x7fda2c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda2bfff700 1 -- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c004810 msgr2=0x7fda2c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda2bfff700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c004810 0x7fda2c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda2bfff700 1 -- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda2c0b0ad0 msgr2=0x7fda2c0c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda2bfff700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda2c0b0ad0 0x7fda2c0c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda2bfff700 1 -- 172.21.15.118:0/851033223 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c166380 con 0x7fda2c0c4b10 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda2bfff700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda2c0c4b10 0x7fda2c167d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0708e0 tx=0x7fda2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:41.448 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda237fe700 1 -- 172.21.15.118:0/851033223 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2400e070 con 0x7fda2c0c4b10 2024-08-18T13:50:41.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda237fe700 1 -- 172.21.15.118:0/851033223 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24011b80 con 0x7fda2c0c4b10 2024-08-18T13:50:41.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c0c5610 con 0x7fda2c0c4b10 2024-08-18T13:50:41.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda237fe700 1 -- 172.21.15.118:0/851033223 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24016470 con 0x7fda2c0c4b10 2024-08-18T13:50:41.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c0c5bd0 con 0x7fda2c0c4b10 2024-08-18T13:50:41.449 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c0713a0 con 0x7fda2c0c4b10 2024-08-18T13:50:41.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.444+0000 7fda237fe700 1 -- 172.21.15.118:0/851033223 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fda240168a0 con 0x7fda2c0c4b10 2024-08-18T13:50:41.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.448+0000 7fda237fe700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda10060890 0x7fda10062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:41.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.448+0000 7fda237fe700 1 -- 172.21.15.118:0/851033223 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fda2407dbf0 con 0x7fda2c0c4b10 2024-08-18T13:50:41.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.448+0000 7fda2b7fe700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda10060890 0x7fda10062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:41.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.448+0000 7fda2b7fe700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda10060890 0x7fda10062d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000c00 tx=0x7fda1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:41.457 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.452+0000 7fda237fe700 1 -- 172.21.15.118:0/851033223 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fda2404dad0 con 0x7fda2c0c4b10 2024-08-18T13:50:41.658 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.656+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda2c06a400 con 0x7fda2c0c4b10 2024-08-18T13:50:41.659 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.656+0000 7fda237fe700 1 -- 172.21.15.118:0/851033223 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fda240510f0 con 0x7fda2c0c4b10 2024-08-18T13:50:41.660 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:41.660 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda10060890 msgr2=0x7fda10062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda10060890 0x7fda10062d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000c00 tx=0x7fda1c009040 comp rx=0 tx=0).stop 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda2c0c4b10 msgr2=0x7fda2c167d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda2c0c4b10 0x7fda2c167d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0708e0 tx=0x7fda2400b040 comp rx=0 tx=0).stop 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 shutdown_connections 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fda10060890 0x7fda10062d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda2c0c4b10 0x7fda2c167d40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda2c0b0ad0 0x7fda2c0c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 --2- 172.21.15.118:0/851033223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda0c004810 0x7fda2c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:41.662 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 >> 172.21.15.118:0/851033223 conn(0x7fda2c0cc600 msgr2=0x7fda2c154090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:41.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 shutdown_connections 2024-08-18T13:50:41.663 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:41.660+0000 7fda3085c700 1 -- 172.21.15.118:0/851033223 wait complete. 2024-08-18T13:50:41.663 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:41.678 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:50:43.679 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:43.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 Processor -- start 2024-08-18T13:50:43.790 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 -- start start 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900c0cf0 0x7f9b90069d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9b90068260 0x7f9b9014f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9b900897f0 0x7f9b9014f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b900c2270 con 0x7f9b900897f0 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b90068d50 con 0x7f9b90068260 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b96d67700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900698b0 con 0x7f9b900c0cf0 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9b900897f0 0x7f9b9014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b8ffff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9b90068260 0x7f9b9014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:43.792 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f9b900897f0 0x7f9b9014f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54380/0 (socket says 172.21.15.118:54380) 2024-08-18T13:50:43.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 -- 172.21.15.118:0/2034451866 learned_addr learned my addr 172.21.15.118:0/2034451866 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:43.793 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b94b03700 1 --2- 172.21.15.118:0/2034451866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900c0cf0 0x7f9b90069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:43.794 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 -- 172.21.15.118:0/2034451866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900c0cf0 msgr2=0x7f9b90069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 --2- 172.21.15.118:0/2034451866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900c0cf0 0x7f9b90069d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 -- 172.21.15.118:0/2034451866 >> v2:172.21.15.178:3300/0 conn(0x7f9b90068260 msgr2=0x7f9b9014f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 --2- 172.21.15.118:0/2034451866 >> v2:172.21.15.178:3300/0 conn(0x7f9b90068260 0x7f9b9014f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 -- 172.21.15.118:0/2034451866 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90156090 con 0x7f9b900897f0 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.788+0000 7f9b95304700 1 --2- 172.21.15.118:0/2034451866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900897f0 0x7f9b9014f670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9b90088b30 tx=0x7f9b88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1700be6807e1d82 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b8f7fe700 1 -- 172.21.15.118:0/2034451866 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800b640 con 0x7f9b900897f0 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b8f7fe700 1 -- 172.21.15.118:0/2034451866 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8800b7a0 con 0x7f9b900897f0 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b8f7fe700 1 -- 172.21.15.118:0/2034451866 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800bb40 con 0x7f9b900897f0 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 -- 172.21.15.118:0/2034451866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900897f0 msgr2=0x7f9b9014f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:43.795 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2034451866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900897f0 0x7f9b9014f670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9b90088b30 tx=0x7f9b88009250 comp rx=0 tx=0).stop 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 -- 172.21.15.118:0/2034451866 shutdown_connections 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2034451866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900897f0 0x7f9b9014f670 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2034451866 >> v2:172.21.15.178:3300/0 conn(0x7f9b90068260 0x7f9b9014f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2034451866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900c0cf0 0x7f9b90069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 -- 172.21.15.118:0/2034451866 >> 172.21.15.118:0/2034451866 conn(0x7f9b90067e60 msgr2=0x7f9b900cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 -- 172.21.15.118:0/2034451866 shutdown_connections 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 -- 172.21.15.118:0/2034451866 wait complete. 2024-08-18T13:50:43.796 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 Processor -- start 2024-08-18T13:50:43.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.792+0000 7f9b96d67700 1 -- start start 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b90068260 0x7f9b900cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900897f0 0x7f9b90154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900c0cf0 0x7f9b9013d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b901532b0 con 0x7f9b900c0cf0 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b901544c0 con 0x7f9b90068260 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b90155aa0 con 0x7f9b900897f0 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900c0cf0 0x7f9b9013d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b8ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900897f0 0x7f9b90154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:43.798 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b8ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900897f0 0x7f9b90154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:39746/0 (socket says 172.21.15.118:39746) 2024-08-18T13:50:43.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900c0cf0 0x7f9b9013d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54394/0 (socket says 172.21.15.118:54394) 2024-08-18T13:50:43.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b8ffff700 1 -- 172.21.15.118:0/2406775245 learned_addr learned my addr 172.21.15.118:0/2406775245 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:43.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b94b03700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b90068260 0x7f9b900cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:43.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 -- 172.21.15.118:0/2406775245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900897f0 msgr2=0x7f9b90154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:43.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900897f0 0x7f9b90154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:43.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 -- 172.21.15.118:0/2406775245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b90068260 msgr2=0x7f9b900cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:43.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b90068260 0x7f9b900cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:43.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 -- 172.21.15.118:0/2406775245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880094f0 con 0x7f9b900c0cf0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b95304700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900c0cf0 0x7f9b9013d190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9b90070980 tx=0x7f9b88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b8cff9700 1 -- 172.21.15.118:0/2406775245 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800b910 con 0x7f9b900c0cf0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b8cff9700 1 -- 172.21.15.118:0/2406775245 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8800ba70 con 0x7f9b900c0cf0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b90153eb0 con 0x7f9b900c0cf0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b8cff9700 1 -- 172.21.15.118:0/2406775245 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88025670 con 0x7f9b900c0cf0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.796+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b90154150 con 0x7f9b900c0cf0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.800+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b90154670 con 0x7f9b900c0cf0 2024-08-18T13:50:43.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.800+0000 7f9b8cff9700 1 -- 172.21.15.118:0/2406775245 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9b88023070 con 0x7f9b900c0cf0 2024-08-18T13:50:43.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.800+0000 7f9b8cff9700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b70060860 0x7f9b70062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:43.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.800+0000 7f9b8cff9700 1 -- 172.21.15.118:0/2406775245 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7f9b8807e570 con 0x7f9b900c0cf0 2024-08-18T13:50:43.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.800+0000 7f9b94b03700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b70060860 0x7f9b70062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:43.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.800+0000 7f9b94b03700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b70060860 0x7f9b70062d10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009db0 tx=0x7f9b780091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:43.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:43.808+0000 7f9b8cff9700 1 -- 172.21.15.118:0/2406775245 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9b90154670 con 0x7f9b900c0cf0 2024-08-18T13:50:44.018 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.016+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b90154670 con 0x7f9b900c0cf0 2024-08-18T13:50:44.019 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.016+0000 7f9b8cff9700 1 -- 172.21.15.118:0/2406775245 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f9b90154670 con 0x7f9b900c0cf0 2024-08-18T13:50:44.019 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:44.019 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:44.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b70060860 msgr2=0x7f9b70062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:44.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b70060860 0x7f9b70062d10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009db0 tx=0x7f9b780091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:44.021 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900c0cf0 msgr2=0x7f9b9013d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900c0cf0 0x7f9b9013d190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9b90070980 tx=0x7f9b88017000 comp rx=0 tx=0).stop 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 shutdown_connections 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9b70060860 0x7f9b70062d10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9b900c0cf0 0x7f9b9013d190 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b900897f0 0x7f9b90154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 --2- 172.21.15.118:0/2406775245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b90068260 0x7f9b900cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 >> 172.21.15.118:0/2406775245 conn(0x7f9b90067e60 msgr2=0x7f9b9014e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 shutdown_connections 2024-08-18T13:50:44.022 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:44.020+0000 7f9b96d67700 1 -- 172.21.15.118:0/2406775245 wait complete. 2024-08-18T13:50:44.023 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:44.033 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:50:44.262 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:50:44.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:50:46.034 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:46.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a759e700 1 Processor -- start 2024-08-18T13:50:46.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a759e700 1 -- start start 2024-08-18T13:50:46.152 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe4a80c8200 0x7fe4a80673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:46.152 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a759e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe4a806f2f0 0x7fe4a80c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:46.152 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a759e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a806ee60 con 0x7fe4a8153690 2024-08-18T13:50:46.152 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a815a640 con 0x7fe4a80c8200 2024-08-18T13:50:46.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a759e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a806f730 con 0x7fe4a806f2f0 2024-08-18T13:50:46.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe49ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe4a80c8200 0x7fe4a80673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:46.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe49ffff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe4a80c8200 0x7fe4a80673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44818/0 (socket says 172.21.15.118:44818) 2024-08-18T13:50:46.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe49ffff700 1 -- 172.21.15.118:0/3488957779 learned_addr learned my addr 172.21.15.118:0/3488957779 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:46.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe4a6d9d700 1 --2- 172.21.15.118:0/3488957779 >> v2:172.21.15.178:3300/0 conn(0x7fe4a806f2f0 0x7fe4a80c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:46.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe49ffff700 1 -- 172.21.15.118:0/3488957779 >> v2:172.21.15.178:3300/0 conn(0x7fe4a806f2f0 msgr2=0x7fe4a80c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:46.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe49ffff700 1 --2- 172.21.15.118:0/3488957779 >> v2:172.21.15.178:3300/0 conn(0x7fe4a806f2f0 0x7fe4a80c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe49ffff700 1 -- 172.21.15.118:0/3488957779 >> v1:172.21.15.178:6789/0 conn(0x7fe4a8153690 legacy=0x7fe4a80c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:46.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.148+0000 7fe49ffff700 1 -- 172.21.15.118:0/3488957779 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8153d10 con 0x7fe4a80c8200 2024-08-18T13:50:46.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe49ffff700 1 --2- 172.21.15.118:0/3488957779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a80c8200 0x7fe4a80673d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806aa00 tx=0x7fe498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503c701c1ed814fe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:46.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a5d9b700 1 -- 172.21.15.118:0/3488957779 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498009e70 con 0x7fe4a80c8200 2024-08-18T13:50:46.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a5d9b700 1 -- 172.21.15.118:0/3488957779 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49800b860 con 0x7fe4a80c8200 2024-08-18T13:50:46.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a5d9b700 1 -- 172.21.15.118:0/3488957779 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800bc00 con 0x7fe4a80c8200 2024-08-18T13:50:46.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 -- 172.21.15.118:0/3488957779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a80c8200 msgr2=0x7fe4a80673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:46.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 --2- 172.21.15.118:0/3488957779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a80c8200 0x7fe4a80673d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806aa00 tx=0x7fe498009250 comp rx=0 tx=0).stop 2024-08-18T13:50:46.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 -- 172.21.15.118:0/3488957779 shutdown_connections 2024-08-18T13:50:46.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 --2- 172.21.15.118:0/3488957779 >> v2:172.21.15.178:3300/0 conn(0x7fe4a806f2f0 0x7fe4a80c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 --2- 172.21.15.118:0/3488957779 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a80c8200 0x7fe4a80673d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 -- 172.21.15.118:0/3488957779 >> 172.21.15.118:0/3488957779 conn(0x7fe4a806c020 msgr2=0x7fe4a80c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:46.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 -- 172.21.15.118:0/3488957779 shutdown_connections 2024-08-18T13:50:46.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 -- 172.21.15.118:0/3488957779 wait complete. 2024-08-18T13:50:46.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 Processor -- start 2024-08-18T13:50:46.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.152+0000 7fe4a759e700 1 -- start start 2024-08-18T13:50:46.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a806f2f0 0x7fe4a814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:46.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4a80c8200 0x7fe4a816acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:46.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4a8153690 0x7fe4a816b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:46.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a816d1a0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a8169850 con 0x7fe4a8153690 2024-08-18T13:50:46.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a816a390 con 0x7fe4a80c8200 2024-08-18T13:50:46.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a806f2f0 0x7fe4a814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a806f2f0 0x7fe4a814fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44830/0 (socket says 172.21.15.118:44830) 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 -- 172.21.15.118:0/2560939969 learned_addr learned my addr 172.21.15.118:0/2560939969 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a6d9d700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4a8153690 0x7fe4a816b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe49ffff700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4a80c8200 0x7fe4a816acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 -- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4a80c8200 msgr2=0x7fe4a816acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4a80c8200 0x7fe4a816acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 -- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4a8153690 msgr2=0x7fe4a816b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4a8153690 0x7fe4a816b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 -- 172.21.15.118:0/2560939969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4980094f0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a6d9d700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4a8153690 0x7fe4a816b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a659c700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a806f2f0 0x7fe4a814fc20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c5c0 tx=0x7fe49000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:46.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe49e7fc700 1 -- 172.21.15.118:0/2560939969 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490011590 con 0x7fe4a806f2f0 2024-08-18T13:50:46.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe49e7fc700 1 -- 172.21.15.118:0/2560939969 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe490011710 con 0x7fe4a806f2f0 2024-08-18T13:50:46.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a816cee0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe49e7fc700 1 -- 172.21.15.118:0/2560939969 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490011ad0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.156+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a8169d40 con 0x7fe4a806f2f0 2024-08-18T13:50:46.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.160+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a80713a0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.160+0000 7fe49e7fc700 1 -- 172.21.15.118:0/2560939969 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe49001b3b0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.160+0000 7fe49e7fc700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe488060890 0x7fe488062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:46.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.160+0000 7fe49e7fc700 1 -- 172.21.15.118:0/2560939969 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fe49007c650 con 0x7fe4a806f2f0 2024-08-18T13:50:46.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.160+0000 7fe49ffff700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe488060890 0x7fe488062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:46.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.160+0000 7fe49ffff700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe488060890 0x7fe488062d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806aa00 tx=0x7fe498017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:46.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.168+0000 7fe49e7fc700 1 -- 172.21.15.118:0/2560939969 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe49004c530 con 0x7fe4a806f2f0 2024-08-18T13:50:46.371 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.368+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4a81594e0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.372 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.368+0000 7fe49e7fc700 1 -- 172.21.15.118:0/2560939969 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fe4a81594e0 con 0x7fe4a806f2f0 2024-08-18T13:50:46.372 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:46.372 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe488060890 msgr2=0x7fe488062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe488060890 0x7fe488062d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806aa00 tx=0x7fe498017000 comp rx=0 tx=0).stop 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a806f2f0 msgr2=0x7fe4a814fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a806f2f0 0x7fe4a814fc20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c5c0 tx=0x7fe49000c8d0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 shutdown_connections 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe488060890 0x7fe488062d40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4a8153690 0x7fe4a816b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4a80c8200 0x7fe4a816acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 --2- 172.21.15.118:0/2560939969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe4a806f2f0 0x7fe4a814fc20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:46.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 >> 172.21.15.118:0/2560939969 conn(0x7fe4a806c020 msgr2=0x7fe4a80c62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:46.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 shutdown_connections 2024-08-18T13:50:46.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:46.372+0000 7fe4a759e700 1 -- 172.21.15.118:0/2560939969 wait complete. 2024-08-18T13:50:46.376 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:46.386 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:50:48.387 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:48.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 Processor -- start 2024-08-18T13:50:48.502 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 -- start start 2024-08-18T13:50:48.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc38c0be710 0x7fc38c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:48.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc38c0c3d50 0x7fc38c0c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:48.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c0c3020 con 0x7fc38c0be340 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0c38c0 con 0x7fc38c0be710 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0beb10 con 0x7fc38c0c3d50 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc38c0be710 0x7fc38c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc38c0be710 0x7fc38c06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44832/0 (socket says 172.21.15.118:44832) 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 -- 172.21.15.118:0/237181681 learned_addr learned my addr 172.21.15.118:0/237181681 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc391928700 1 --2- 172.21.15.118:0/237181681 >> v2:172.21.15.178:3300/0 conn(0x7fc38c0c3d50 0x7fc38c0c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 -- 172.21.15.118:0/237181681 >> v2:172.21.15.178:3300/0 conn(0x7fc38c0c3d50 msgr2=0x7fc38c0c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 --2- 172.21.15.118:0/237181681 >> v2:172.21.15.178:3300/0 conn(0x7fc38c0c3d50 0x7fc38c0c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 -- 172.21.15.118:0/237181681 >> v1:172.21.15.178:6789/0 conn(0x7fc38c0be340 legacy=0x7fc38c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 -- 172.21.15.118:0/237181681 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c06e3b0 con 0x7fc38c0be710 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc391928700 1 --2- 172.21.15.118:0/237181681 >> v2:172.21.15.178:3300/0 conn(0x7fc38c0c3d50 0x7fc38c0c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:50:48.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc390926700 1 --2- 172.21.15.118:0/237181681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c06db20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc38c06d4f0 tx=0x7fc384009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2234dce0f2bd3f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:48.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc383fff700 1 -- 172.21.15.118:0/237181681 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38400e070 con 0x7fc38c0be710 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc383fff700 1 -- 172.21.15.118:0/237181681 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38400ba10 con 0x7fc38c0be710 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc383fff700 1 -- 172.21.15.118:0/237181681 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3840092e0 con 0x7fc38c0be710 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 -- 172.21.15.118:0/237181681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 msgr2=0x7fc38c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 --2- 172.21.15.118:0/237181681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c06db20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc38c06d4f0 tx=0x7fc384009840 comp rx=0 tx=0).stop 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 -- 172.21.15.118:0/237181681 shutdown_connections 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 --2- 172.21.15.118:0/237181681 >> v2:172.21.15.178:3300/0 conn(0x7fc38c0c3d50 0x7fc38c0c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 --2- 172.21.15.118:0/237181681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c06db20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.500+0000 7fc39338b700 1 -- 172.21.15.118:0/237181681 >> 172.21.15.118:0/237181681 conn(0x7fc38c0c7c40 msgr2=0x7fc38c0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 -- 172.21.15.118:0/237181681 shutdown_connections 2024-08-18T13:50:48.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 -- 172.21.15.118:0/237181681 wait complete. 2024-08-18T13:50:48.507 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 Processor -- start 2024-08-18T13:50:48.507 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 -- start start 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38c0be340 0x7fc38c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc38c0c3d50 0x7fc38c16caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c151e60 con 0x7fc38c0be710 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c154460 con 0x7fc38c0c3d50 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.504+0000 7fc39338b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c15ca80 con 0x7fc38c0be340 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c15cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44838/0 (socket says 172.21.15.118:44838) 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 -- 172.21.15.118:0/3629927456 learned_addr learned my addr 172.21.15.118:0/3629927456 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc391928700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc38c0c3d50 0x7fc38c16caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc391127700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38c0be340 0x7fc38c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 -- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38c0be340 msgr2=0x7fc38c159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:48.511 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38c0be340 0x7fc38c159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 -- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc38c0c3d50 msgr2=0x7fc38c16caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:48.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc38c0c3d50 0x7fc38c16caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 -- 172.21.15.118:0/3629927456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc384009ae0 con 0x7fc38c0be710 2024-08-18T13:50:48.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc391928700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc38c0c3d50 0x7fc38c16caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:50:48.512 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc390926700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c15cf10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc38c06d1d0 tx=0x7fc384017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:48.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc3817fa700 1 -- 172.21.15.118:0/3629927456 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384012ac0 con 0x7fc38c0be710 2024-08-18T13:50:48.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc3817fa700 1 -- 172.21.15.118:0/3629927456 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc384012c60 con 0x7fc38c0be710 2024-08-18T13:50:48.513 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c1541c0 con 0x7fc38c0be710 2024-08-18T13:50:48.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc3817fa700 1 -- 172.21.15.118:0/3629927456 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38401d420 con 0x7fc38c0be710 2024-08-18T13:50:48.514 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c1596c0 con 0x7fc38c0be710 2024-08-18T13:50:48.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc3817fa700 1 -- 172.21.15.118:0/3629927456 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc38401d7d0 con 0x7fc38c0be710 2024-08-18T13:50:48.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.508+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38c15c850 con 0x7fc38c0be710 2024-08-18T13:50:48.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.512+0000 7fc3817fa700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc370060890 0x7fc370062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:48.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.512+0000 7fc3817fa700 1 -- 172.21.15.118:0/3629927456 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7089+0+0 (secure 0 0 0) 0x7fc38407e950 con 0x7fc38c0be710 2024-08-18T13:50:48.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.512+0000 7fc391127700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc370060890 0x7fc370062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:48.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.512+0000 7fc391127700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc370060890 0x7fc370062d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc37c003870 tx=0x7fc37c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:48.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.520+0000 7fc3817fa700 1 -- 172.21.15.118:0/3629927456 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc38c15c850 con 0x7fc38c0be710 2024-08-18T13:50:48.723 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.720+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc38c15c850 con 0x7fc38c0be710 2024-08-18T13:50:48.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.720+0000 7fc3817fa700 1 -- 172.21.15.118:0/3629927456 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6130 (secure 0 0 0) 0x7fc38c15c850 con 0x7fc38c0be710 2024-08-18T13:50:48.724 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:48.724 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:49:56.741103+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5464":{"gid":5464,"name":"f","rank":0,"incarnation":31,"state":"up:active","state_seq":37,"addr":"172.21.15.178:6835/2054758182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2054758182},{"type":"v1","addr":"172.21.15.178:6835","nonce":2054758182}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:48.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc370060890 msgr2=0x7fc370062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc370060890 0x7fc370062d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc37c003870 tx=0x7fc37c009040 comp rx=0 tx=0).stop 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 msgr2=0x7fc38c15cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c15cf10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc38c06d1d0 tx=0x7fc384017000 comp rx=0 tx=0).stop 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 shutdown_connections 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc370060890 0x7fc370062d40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc38c0c3d50 0x7fc38c16caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc38c0be710 0x7fc38c15cf10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 --2- 172.21.15.118:0/3629927456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38c0be340 0x7fc38c159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 >> 172.21.15.118:0/3629927456 conn(0x7fc38c0c7c40 msgr2=0x7fc38c06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 shutdown_connections 2024-08-18T13:50:48.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:48.724+0000 7fc39338b700 1 -- 172.21.15.118:0/3629927456 wait complete. 2024-08-18T13:50:48.728 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 36 2024-08-18T13:50:48.738 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-08-18T13:50:50.739 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:50.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff850df4700 1 Processor -- start 2024-08-18T13:50:50.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff850df4700 1 -- start start 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff850df4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0add40 0x7ff84c155b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff850df4700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff84c0ae970 0x7ff84c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff850df4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84c156060 con 0x7ff84c0b15f0 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff850df4700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84c155110 con 0x7ff84c0ae970 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff850df4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c06f340 con 0x7ff84c0add40 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff84c0ae970 0x7ff84c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff84c0ae970 0x7ff84c15fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44850/0 (socket says 172.21.15.118:44850) 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84bfff700 1 -- 172.21.15.118:0/2398774012 learned_addr learned my addr 172.21.15.118:0/2398774012 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84b7fe700 1 --2- 172.21.15.118:0/2398774012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0add40 0x7ff84c155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84bfff700 1 -- 172.21.15.118:0/2398774012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0add40 msgr2=0x7ff84c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84bfff700 1 --2- 172.21.15.118:0/2398774012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0add40 0x7ff84c155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:50.833 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84bfff700 1 -- 172.21.15.118:0/2398774012 >> v1:172.21.15.178:6789/0 conn(0x7ff84c0b15f0 legacy=0x7ff84c0c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:50.834 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.828+0000 7ff84bfff700 1 -- 172.21.15.118:0/2398774012 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c0aeeb0 con 0x7ff84c0ae970 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff84bfff700 1 --2- 172.21.15.118:0/2398774012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0ae970 0x7ff84c15fab0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff844009f20 tx=0x7ff844009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4630fa66da4c23b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff84a7fc700 1 -- 172.21.15.118:0/2398774012 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84400e070 con 0x7ff84c0ae970 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff84a7fc700 1 -- 172.21.15.118:0/2398774012 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84400ba60 con 0x7ff84c0ae970 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff84a7fc700 1 -- 172.21.15.118:0/2398774012 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440092e0 con 0x7ff84c0ae970 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 -- 172.21.15.118:0/2398774012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0ae970 msgr2=0x7ff84c15fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 --2- 172.21.15.118:0/2398774012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0ae970 0x7ff84c15fab0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff844009f20 tx=0x7ff844009840 comp rx=0 tx=0).stop 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 -- 172.21.15.118:0/2398774012 shutdown_connections 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 --2- 172.21.15.118:0/2398774012 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0ae970 0x7ff84c15fab0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 --2- 172.21.15.118:0/2398774012 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0add40 0x7ff84c155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 -- 172.21.15.118:0/2398774012 >> 172.21.15.118:0/2398774012 conn(0x7ff84c0cc600 msgr2=0x7ff84c0c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:50.835 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 -- 172.21.15.118:0/2398774012 shutdown_connections 2024-08-18T13:50:50.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 -- 172.21.15.118:0/2398774012 wait complete. 2024-08-18T13:50:50.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 Processor -- start 2024-08-18T13:50:50.836 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 -- start start 2024-08-18T13:50:50.838 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.832+0000 7ff850df4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c0add40 0x7ff84c0c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:50.838 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0ae970 0x7ff84c0c0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:50.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0b15f0 0x7ff84c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:50.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c7500 con 0x7ff84c0b15f0 2024-08-18T13:50:50.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0bf5e0 con 0x7ff84c0add40 2024-08-18T13:50:50.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0c1ee0 con 0x7ff84c0ae970 2024-08-18T13:50:50.839 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0b15f0 0x7ff84c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0b15f0 0x7ff84c14ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44858/0 (socket says 172.21.15.118:44858) 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 -- 172.21.15.118:0/1929887633 learned_addr learned my addr 172.21.15.118:0/1929887633 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84b7fe700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c0add40 0x7ff84c0c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84affd700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0ae970 0x7ff84c0c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 -- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0ae970 msgr2=0x7ff84c0c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0ae970 0x7ff84c0c0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 -- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c0add40 msgr2=0x7ff84c0c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c0add40 0x7ff84c0c0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 -- 172.21.15.118:0/1929887633 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844009ae0 con 0x7ff84c0b15f0 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84affd700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0ae970 0x7ff84c0c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff84bfff700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0b15f0 0x7ff84c14ccc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff844009880 tx=0x7ff844018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:50.840 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff8337fe700 1 -- 172.21.15.118:0/1929887633 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844008d90 con 0x7ff84c0b15f0 2024-08-18T13:50:50.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff8337fe700 1 -- 172.21.15.118:0/1929887633 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff844026460 con 0x7ff84c0b15f0 2024-08-18T13:50:50.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c0c21c0 con 0x7ff84c0b15f0 2024-08-18T13:50:50.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff8337fe700 1 -- 172.21.15.118:0/1929887633 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844026820 con 0x7ff84c0b15f0 2024-08-18T13:50:50.841 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c0bf830 con 0x7ff84c0b15f0 2024-08-18T13:50:50.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c0c2640 con 0x7ff84c0b15f0 2024-08-18T13:50:50.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff8337fe700 1 -- 172.21.15.118:0/1929887633 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff844003040 con 0x7ff84c0b15f0 2024-08-18T13:50:50.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.836+0000 7ff8337fe700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff838060860 0x7ff838062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:50.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.840+0000 7ff8337fe700 1 -- 172.21.15.118:0/1929887633 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7ff84407f550 con 0x7ff84c0b15f0 2024-08-18T13:50:50.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.840+0000 7ff84b7fe700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff838060860 0x7ff838062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:50.842 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.840+0000 7ff84b7fe700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff838060860 0x7ff838062d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009db0 tx=0x7ff83c0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:50.850 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:50.848+0000 7ff8337fe700 1 -- 172.21.15.118:0/1929887633 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff84404f3e0 con 0x7ff84c0b15f0 2024-08-18T13:50:51.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.044+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff84c0713a0 con 0x7ff84c0b15f0 2024-08-18T13:50:51.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.044+0000 7ff8337fe700 1 -- 172.21.15.118:0/1929887633 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5369 (secure 0 0 0) 0x7ff844052a00 con 0x7ff84c0b15f0 2024-08-18T13:50:51.047 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:51.047 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:50.639803+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:51.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff838060860 msgr2=0x7ff838062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.049 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff838060860 0x7ff838062d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009db0 tx=0x7ff83c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0b15f0 msgr2=0x7ff84c14ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0b15f0 0x7ff84c14ccc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff844009880 tx=0x7ff844018000 comp rx=0 tx=0).stop 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 shutdown_connections 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff838060860 0x7ff838062d10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff84c0b15f0 0x7ff84c14ccc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff84c0ae970 0x7ff84c0c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 --2- 172.21.15.118:0/1929887633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c0add40 0x7ff84c0c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 >> 172.21.15.118:0/1929887633 conn(0x7ff84c0cc600 msgr2=0x7ff84c14ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 shutdown_connections 2024-08-18T13:50:51.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.048+0000 7ff850df4700 1 -- 172.21.15.118:0/1929887633 wait complete. 2024-08-18T13:50:51.051 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 37 2024-08-18T13:50:51.061 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-08-18T13:50:51.061 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-08-18T13:50:51.062 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:51.168 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.164+0000 7f86eef4b700 1 Processor -- start 2024-08-18T13:50:51.168 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.164+0000 7f86eef4b700 1 -- start start 2024-08-18T13:50:51.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.164+0000 7f86eef4b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e80684e0 0x7f86e8155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:51.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.164+0000 7f86eef4b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f86e80be4b0 0x7f86e8156000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:51.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.164+0000 7f86eef4b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80c1110 con 0x7f86e81556f0 2024-08-18T13:50:51.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.164+0000 7f86eef4b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e80c0b90 con 0x7f86e80be4b0 2024-08-18T13:50:51.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.164+0000 7f86eef4b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80c1cb0 con 0x7f86e80684e0 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f86e80be4b0 0x7f86e8156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ed748700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e80684e0 0x7f86e8155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f86e80be4b0 0x7f86e8156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:53514/0 (socket says 172.21.15.118:53514) 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 -- 172.21.15.118:0/208310139 learned_addr learned my addr 172.21.15.118:0/208310139 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86edf49700 1 --1- 172.21.15.118:0/208310139 >> v1:172.21.15.178:6789/0 conn(0x7f86e81556f0 0x7f86e80c15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:52284/0 (socket says 172.21.15.118:52284) 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 -- 172.21.15.118:0/208310139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e80684e0 msgr2=0x7f86e8155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 --2- 172.21.15.118:0/208310139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e80684e0 0x7f86e8155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 -- 172.21.15.118:0/208310139 >> v1:172.21.15.178:6789/0 conn(0x7f86e81556f0 legacy=0x7f86e80c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 -- 172.21.15.118:0/208310139 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80c8060 con 0x7f86e80be4b0 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ee74a700 1 --2- 172.21.15.118:0/208310139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80be4b0 0x7f86e8156000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86e000d1d0 tx=0x7f86e000ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1291f2a40d5a641b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ecf47700 1 -- 172.21.15.118:0/208310139 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0010070 con 0x7f86e80be4b0 2024-08-18T13:50:51.171 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ecf47700 1 -- 172.21.15.118:0/208310139 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e000de00 con 0x7f86e80be4b0 2024-08-18T13:50:51.172 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86ecf47700 1 -- 172.21.15.118:0/208310139 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e000b320 con 0x7f86e80be4b0 2024-08-18T13:50:51.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 -- 172.21.15.118:0/208310139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80be4b0 msgr2=0x7f86e8156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 --2- 172.21.15.118:0/208310139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80be4b0 0x7f86e8156000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86e000d1d0 tx=0x7f86e000ba50 comp rx=0 tx=0).stop 2024-08-18T13:50:51.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 -- 172.21.15.118:0/208310139 shutdown_connections 2024-08-18T13:50:51.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 --2- 172.21.15.118:0/208310139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80be4b0 0x7f86e8156000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 --2- 172.21.15.118:0/208310139 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e80684e0 0x7f86e8155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 -- 172.21.15.118:0/208310139 >> 172.21.15.118:0/208310139 conn(0x7f86e806c020 msgr2=0x7f86e80c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:51.177 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 -- 172.21.15.118:0/208310139 shutdown_connections 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 -- 172.21.15.118:0/208310139 wait complete. 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 Processor -- start 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.168+0000 7f86eef4b700 1 -- start start 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80684e0 0x7f86e813d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86e80be4b0 0x7f86e8164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e81556f0 0x7f86e8148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8149610 con 0x7f86e80be4b0 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e81493f0 con 0x7f86e80684e0 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e813d290 con 0x7f86e81556f0 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86ed748700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86e80be4b0 0x7f86e8164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80684e0 0x7f86e813d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80684e0 0x7f86e813d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:53516/0 (socket says 172.21.15.118:53516) 2024-08-18T13:50:51.178 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 -- 172.21.15.118:0/251497498 learned_addr learned my addr 172.21.15.118:0/251497498 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86ee74a700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e81556f0 0x7f86e8148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 -- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e81556f0 msgr2=0x7f86e8148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e81556f0 0x7f86e8148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 -- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86e80be4b0 msgr2=0x7f86e8164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86e80be4b0 0x7f86e8164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 -- 172.21.15.118:0/251497498 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e000bcf0 con 0x7f86e80684e0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86ee74a700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e81556f0 0x7f86e8148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86edf49700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80684e0 0x7f86e813d720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c560 tx=0x7f86e4007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86ddffb700 1 -- 172.21.15.118:0/251497498 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4005a70 con 0x7f86e80684e0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86ddffb700 1 -- 172.21.15.118:0/251497498 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4005bd0 con 0x7f86e80684e0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e80c7490 con 0x7f86e80684e0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86ddffb700 1 -- 172.21.15.118:0/251497498 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40070d0 con 0x7f86e80684e0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.172+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e813cf60 con 0x7f86e80684e0 2024-08-18T13:50:51.179 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.176+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e80713a0 con 0x7f86e80684e0 2024-08-18T13:50:51.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.176+0000 7f86ddffb700 1 -- 172.21.15.118:0/251497498 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f86e4007480 con 0x7f86e80684e0 2024-08-18T13:50:51.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.176+0000 7f86ddffb700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f86cc060860 0x7f86cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:51.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.176+0000 7f86ddffb700 1 -- 172.21.15.118:0/251497498 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f86e407cb60 con 0x7f86e80684e0 2024-08-18T13:50:51.180 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.176+0000 7f86ed748700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f86cc060860 0x7f86cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:51.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.176+0000 7f86ed748700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f86cc060860 0x7f86cc062d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f86e80bf8b0 tx=0x7f86d80091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:51.190 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.188+0000 7f86ddffb700 1 -- 172.21.15.118:0/251497498 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f86e404ca70 con 0x7f86e80684e0 2024-08-18T13:50:51.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.388+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86e806ed30 con 0x7f86e80684e0 2024-08-18T13:50:51.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.392+0000 7f86ddffb700 1 -- 172.21.15.118:0/251497498 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f86e4017020 con 0x7f86e80684e0 2024-08-18T13:50:51.394 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:51.394 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:50.639803+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f86cc060860 msgr2=0x7f86cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f86cc060860 0x7f86cc062d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f86e80bf8b0 tx=0x7f86d80091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80684e0 msgr2=0x7f86e813d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80684e0 0x7f86e813d720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c560 tx=0x7f86e4007f70 comp rx=0 tx=0).stop 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 shutdown_connections 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f86cc060860 0x7f86cc062d10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f86e81556f0 0x7f86e8148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f86e80be4b0 0x7f86e8164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.398 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 --2- 172.21.15.118:0/251497498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86e80684e0 0x7f86e813d720 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:51.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 >> 172.21.15.118:0/251497498 conn(0x7f86e806c020 msgr2=0x7f86e806efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:51.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 shutdown_connections 2024-08-18T13:50:51.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:51.396+0000 7f86eef4b700 1 -- 172.21.15.118:0/251497498 wait complete. 2024-08-18T13:50:51.399 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 37 2024-08-18T13:50:51.414 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 37, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 37, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5195": { "addr": "172.21.15.178:6837/3317475884", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6836", "nonce": 3317475884, "type": "v2" }, { "addr": "172.21.15.178:6837", "nonce": 3317475884, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5195, "incarnation": 37, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:50:50.639803+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5195 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:50:53.416 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:53.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.528+0000 7f2e0f380700 1 Processor -- start 2024-08-18T13:50:53.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- start start 2024-08-18T13:50:53.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e080c4780 0x7f2e080c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:53.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e080c3730 con 0x7f2e0814f4a0 2024-08-18T13:50:53.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e080c42d0 con 0x7f2e081552d0 2024-08-18T13:50:53.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e08155830 con 0x7f2e080c4780 2024-08-18T13:50:53.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0e37e700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2e0814f4a0 0x7f2e080c9300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:47950/0 (socket says 172.21.15.118:47950) 2024-08-18T13:50:53.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0e37e700 1 -- 172.21.15.118:0/1656067074 learned_addr learned my addr 172.21.15.118:0/1656067074 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:50:53.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0eb7f700 1 --2- 172.21.15.118:0/1656067074 >> v2:172.21.15.178:3300/0 conn(0x7f2e080c4780 0x7f2e080c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:53.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0d37c700 1 -- 172.21.15.118:0/1656067074 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254736281 0 0) 0x7f2e080c3730 con 0x7f2e0814f4a0 2024-08-18T13:50:53.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0d37c700 1 -- 172.21.15.118:0/1656067074 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df4003570 con 0x7f2e0814f4a0 2024-08-18T13:50:53.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0eb7f700 1 -- 172.21.15.118:0/1656067074 >> v1:172.21.15.178:6789/0 conn(0x7f2e081552d0 legacy=0x7f2e080c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:53.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0eb7f700 1 -- 172.21.15.118:0/1656067074 >> v1:172.21.15.118:6789/0 conn(0x7f2e0814f4a0 legacy=0x7f2e080c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:53.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0eb7f700 1 -- 172.21.15.118:0/1656067074 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e08155b10 con 0x7f2e080c4780 2024-08-18T13:50:53.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0eb7f700 1 --2- 172.21.15.118:0/1656067074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e080c4780 0x7f2e080c4bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009e60 tx=0x7f2e00009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db557f5826ebfa6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:53.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0d37c700 1 -- 172.21.15.118:0/1656067074 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0000e070 con 0x7f2e080c4780 2024-08-18T13:50:53.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0d37c700 1 -- 172.21.15.118:0/1656067074 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0000bb60 con 0x7f2e080c4780 2024-08-18T13:50:53.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0d37c700 1 -- 172.21.15.118:0/1656067074 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00012430 con 0x7f2e080c4780 2024-08-18T13:50:53.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- 172.21.15.118:0/1656067074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e080c4780 msgr2=0x7f2e080c4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:53.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1656067074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e080c4780 0x7f2e080c4bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009e60 tx=0x7f2e00009780 comp rx=0 tx=0).stop 2024-08-18T13:50:53.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- 172.21.15.118:0/1656067074 shutdown_connections 2024-08-18T13:50:53.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1656067074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e080c4780 0x7f2e080c4bc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- 172.21.15.118:0/1656067074 >> 172.21.15.118:0/1656067074 conn(0x7f2e0806f570 msgr2=0x7f2e080c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:53.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- 172.21.15.118:0/1656067074 shutdown_connections 2024-08-18T13:50:53.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.532+0000 7f2e0f380700 1 -- 172.21.15.118:0/1656067074 wait complete. 2024-08-18T13:50:53.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 Processor -- start 2024-08-18T13:50:53.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 -- start start 2024-08-18T13:50:53.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e080c4780 0x7f2e08068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:53.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e0814f4a0 0x7f2e080c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:53.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e081552d0 0x7f2e08146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:53.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080c1820 con 0x7f2e080c4780 2024-08-18T13:50:53.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080c2390 con 0x7f2e081552d0 2024-08-18T13:50:53.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080c77e0 con 0x7f2e0814f4a0 2024-08-18T13:50:53.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e080c4780 0x7f2e08068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:53.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0eb7f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e081552d0 0x7f2e08146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:53.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0eb7f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e081552d0 0x7f2e08146db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:53530/0 (socket says 172.21.15.118:53530) 2024-08-18T13:50:53.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e080c4780 0x7f2e08068010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44868/0 (socket says 172.21.15.118:44868) 2024-08-18T13:50:53.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0eb7f700 1 -- 172.21.15.118:0/1486651613 learned_addr learned my addr 172.21.15.118:0/1486651613 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:53.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0db7d700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e0814f4a0 0x7f2e080c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:53.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 -- 172.21.15.118:0/1486651613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e0814f4a0 msgr2=0x7f2e080c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:53.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e0814f4a0 0x7f2e080c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 -- 172.21.15.118:0/1486651613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e081552d0 msgr2=0x7f2e08146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e081552d0 0x7f2e08146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 -- 172.21.15.118:0/1486651613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e00009a20 con 0x7f2e080c4780 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0e37e700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e080c4780 0x7f2e08068010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c560 tx=0x7f2e04007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2dfe7fc700 1 -- 172.21.15.118:0/1486651613 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04005b30 con 0x7f2e080c4780 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2dfe7fc700 1 -- 172.21.15.118:0/1486651613 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e04005cb0 con 0x7f2e080c4780 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e080c7ac0 con 0x7f2e080c4780 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2dfe7fc700 1 -- 172.21.15.118:0/1486651613 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04007320 con 0x7f2e080c4780 2024-08-18T13:50:53.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.536+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e08165d20 con 0x7f2e080c4780 2024-08-18T13:50:53.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.540+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e080713a0 con 0x7f2e080c4780 2024-08-18T13:50:53.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.540+0000 7f2dfe7fc700 1 -- 172.21.15.118:0/1486651613 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2e0401b3b0 con 0x7f2e080c4780 2024-08-18T13:50:53.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.540+0000 7f2dfe7fc700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2dec060860 0x7f2dec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:53.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.540+0000 7f2dfe7fc700 1 -- 172.21.15.118:0/1486651613 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f2e0407c530 con 0x7f2e080c4780 2024-08-18T13:50:53.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.540+0000 7f2e0db7d700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2dec060860 0x7f2dec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:53.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.540+0000 7f2e0db7d700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2dec060860 0x7f2dec062d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2e080bff20 tx=0x7f2df8007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:53.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.548+0000 7f2dfe7fc700 1 -- 172.21.15.118:0/1486651613 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2e0404c3c0 con 0x7f2e080c4780 2024-08-18T13:50:53.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.752+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e081595c0 con 0x7f2e080c4780 2024-08-18T13:50:53.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.752+0000 7f2dfe7fc700 1 -- 172.21.15.118:0/1486651613 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f2e0404f9e0 con 0x7f2e080c4780 2024-08-18T13:50:53.755 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:53.755 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:50.639803+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:53.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.752+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2dec060860 msgr2=0x7f2dec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.752+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2dec060860 0x7f2dec062d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2e080bff20 tx=0x7f2df8007ec0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.752+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e080c4780 msgr2=0x7f2e08068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e080c4780 0x7f2e08068010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c560 tx=0x7f2e04007d50 comp rx=0 tx=0).stop 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 shutdown_connections 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2dec060860 0x7f2dec062d10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e081552d0 0x7f2e08146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e0814f4a0 0x7f2e080c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 --2- 172.21.15.118:0/1486651613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2e080c4780 0x7f2e08068010 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 >> 172.21.15.118:0/1486651613 conn(0x7f2e0806f570 msgr2=0x7f2e080c30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 shutdown_connections 2024-08-18T13:50:53.758 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:53.756+0000 7f2e0f380700 1 -- 172.21.15.118:0/1486651613 wait complete. 2024-08-18T13:50:53.759 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 37 2024-08-18T13:50:53.769 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:50:55.770 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:55.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.879+0000 7f4edc800700 1 Processor -- start 2024-08-18T13:50:55.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.879+0000 7f4edc800700 1 -- start start 2024-08-18T13:50:55.885 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4ed806c6a0 0x7f4ed8068720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806ca70 0x7f4ed814ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed8068290 con 0x7f4ed80c4fc0 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed8067f60 con 0x7f4ed806c6a0 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8159430 con 0x7f4ed806ca70 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed7fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f4ed80c4fc0 0x7f4ed814f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:56284/0 (socket says 172.21.15.118:56284) 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed77fe700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4ed806c6a0 0x7f4ed8068720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed7fff700 1 -- 172.21.15.118:0/792533983 learned_addr learned my addr 172.21.15.118:0/792533983 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:50:55.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4eceffd700 1 --2- 172.21.15.118:0/792533983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806ca70 0x7f4ed814ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed6ffd700 1 -- 172.21.15.118:0/792533983 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037651912 0 0) 0x7f4ed8068290 con 0x7f4ed80c4fc0 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed6ffd700 1 -- 172.21.15.118:0/792533983 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec4003570 con 0x7f4ed80c4fc0 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed77fe700 1 -- 172.21.15.118:0/792533983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806ca70 msgr2=0x7f4ed814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed77fe700 1 --2- 172.21.15.118:0/792533983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806ca70 0x7f4ed814ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed77fe700 1 -- 172.21.15.118:0/792533983 >> v1:172.21.15.118:6789/0 conn(0x7f4ed80c4fc0 legacy=0x7f4ed814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed77fe700 1 -- 172.21.15.118:0/792533983 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80c9280 con 0x7f4ed806c6a0 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4eceffd700 1 --2- 172.21.15.118:0/792533983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806ca70 0x7f4ed814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed77fe700 1 --2- 172.21.15.118:0/792533983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806c6a0 0x7f4ed8068720 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80098e0 tx=0x7f4ec80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b324290ac9774320 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:55.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed6ffd700 1 -- 172.21.15.118:0/792533983 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8011630 con 0x7f4ed806c6a0 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed6ffd700 1 -- 172.21.15.118:0/792533983 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec8011790 con 0x7f4ed806c6a0 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4ed6ffd700 1 -- 172.21.15.118:0/792533983 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8011b30 con 0x7f4ed806c6a0 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- 172.21.15.118:0/792533983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806c6a0 msgr2=0x7f4ed8068720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 --2- 172.21.15.118:0/792533983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806c6a0 0x7f4ed8068720 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80098e0 tx=0x7f4ec80091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- 172.21.15.118:0/792533983 shutdown_connections 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 --2- 172.21.15.118:0/792533983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806ca70 0x7f4ed814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 --2- 172.21.15.118:0/792533983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806c6a0 0x7f4ed8068720 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- 172.21.15.118:0/792533983 >> 172.21.15.118:0/792533983 conn(0x7f4ed806f4d0 msgr2=0x7f4ed80c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- 172.21.15.118:0/792533983 shutdown_connections 2024-08-18T13:50:55.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.883+0000 7f4edc800700 1 -- 172.21.15.118:0/792533983 wait complete. 2024-08-18T13:50:55.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 Processor -- start 2024-08-18T13:50:55.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 -- start start 2024-08-18T13:50:55.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806c6a0 0x7f4ed8150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:55.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806ca70 0x7f4ed814b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:55.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed80c4fc0 0x7f4ed814b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:55.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed80bf530 con 0x7f4ed806ca70 2024-08-18T13:50:55.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed80bed20 con 0x7f4ed80c4fc0 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed80bef70 con 0x7f4ed806c6a0 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806ca70 0x7f4ed814b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806ca70 0x7f4ed814b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39466/0 (socket says 172.21.15.118:39466) 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 -- 172.21.15.118:0/1615678225 learned_addr learned my addr 172.21.15.118:0/1615678225 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4ed77fe700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806c6a0 0x7f4ed8150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4ed7fff700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed80c4fc0 0x7f4ed814b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 -- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806c6a0 msgr2=0x7f4ed8150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806c6a0 0x7f4ed8150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 -- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed80c4fc0 msgr2=0x7f4ed814b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed80c4fc0 0x7f4ed814b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 -- 172.21.15.118:0/1615678225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009420 con 0x7f4ed806ca70 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4eceffd700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806ca70 0x7f4ed814b350 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000ecb0 tx=0x7f4ec00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4ecffff700 1 -- 172.21.15.118:0/1615678225 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000c920 con 0x7f4ed806ca70 2024-08-18T13:50:55.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4ecffff700 1 -- 172.21.15.118:0/1615678225 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec000ca80 con 0x7f4ed806ca70 2024-08-18T13:50:55.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec8009420 con 0x7f4ed806ca70 2024-08-18T13:50:55.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4ecffff700 1 -- 172.21.15.118:0/1615678225 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000ce20 con 0x7f4ed806ca70 2024-08-18T13:50:55.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.887+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed81502b0 con 0x7f4ed806ca70 2024-08-18T13:50:55.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.891+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed8150110 con 0x7f4ed806ca70 2024-08-18T13:50:55.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.891+0000 7f4ecffff700 1 -- 172.21.15.118:0/1615678225 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4ec0013680 con 0x7f4ed806ca70 2024-08-18T13:50:55.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.891+0000 7f4ecffff700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eb8060860 0x7f4eb8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:55.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.891+0000 7f4ecffff700 1 -- 172.21.15.118:0/1615678225 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f4ec007d430 con 0x7f4ed806ca70 2024-08-18T13:50:55.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.891+0000 7f4ed77fe700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eb8060860 0x7f4eb8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:55.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.895+0000 7f4ed77fe700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eb8060860 0x7f4eb8062d10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80091e0 tx=0x7f4ec8017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:55.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:55.903+0000 7f4ecffff700 1 -- 172.21.15.118:0/1615678225 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4ed8150110 con 0x7f4ed806ca70 2024-08-18T13:50:56.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ed8150110 con 0x7f4ed806ca70 2024-08-18T13:50:56.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4ecffff700 1 -- 172.21.15.118:0/1615678225 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f4ed8150110 con 0x7f4ed806ca70 2024-08-18T13:50:56.106 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:56.107 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:50.639803+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:56.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eb8060860 msgr2=0x7f4eb8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:56.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eb8060860 0x7f4eb8062d10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80091e0 tx=0x7f4ec8017000 comp rx=0 tx=0).stop 2024-08-18T13:50:56.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806ca70 msgr2=0x7f4ed814b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:56.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806ca70 0x7f4ed814b350 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000ecb0 tx=0x7f4ec00091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 shutdown_connections 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4eb8060860 0x7f4eb8062d10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed80c4fc0 0x7f4ed814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ed806ca70 0x7f4ed814b350 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 --2- 172.21.15.118:0/1615678225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ed806c6a0 0x7f4ed8150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.103+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 >> 172.21.15.118:0/1615678225 conn(0x7f4ed806f4d0 msgr2=0x7f4ed80c46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.107+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 shutdown_connections 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:56.107+0000 7f4edc800700 1 -- 172.21.15.118:0/1615678225 wait complete. 2024-08-18T13:50:56.110 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 37 2024-08-18T13:50:56.121 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:50:58.125 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:50:58.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615447e700 1 Processor -- start 2024-08-18T13:50:58.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615447e700 1 -- start start 2024-08-18T13:50:58.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615447e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f614c06b620 0x7f614c06b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615447e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615447e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c06bf30 con 0x7f614c0c2630 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615447e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c068ac0 con 0x7f614c06b620 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615447e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c153160 con 0x7f614c0c4030 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f615221a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f614c06b620 0x7f614c06b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f6152a1b700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f614c0c2630 0x7f614c14c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:38692/0 (socket says 172.21.15.118:38692) 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f6151a19700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:58.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.239+0000 7f6152a1b700 1 -- 172.21.15.118:0/445036167 learned_addr learned my addr 172.21.15.118:0/445036167 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:50:58.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615221a700 1 -- 172.21.15.118:0/445036167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 msgr2=0x7f614c152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:50:58.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615221a700 1 --2- 172.21.15.118:0/445036167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615221a700 1 -- 172.21.15.118:0/445036167 >> v1:172.21.15.178:6789/0 conn(0x7f614c0c2630 legacy=0x7f614c14c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:58.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615221a700 1 -- 172.21.15.118:0/445036167 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c068580 con 0x7f614c06b620 2024-08-18T13:50:58.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615221a700 1 --2- 172.21.15.118:0/445036167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c06b620 0x7f614c06b9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f613c0098e0 tx=0x7f613c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebab64dfd42d8ca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:58.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f6151218700 1 -- 172.21.15.118:0/445036167 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00b640 con 0x7f614c06b620 2024-08-18T13:50:58.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f6151218700 1 -- 172.21.15.118:0/445036167 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c00b7a0 con 0x7f614c06b620 2024-08-18T13:50:58.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f6151218700 1 -- 172.21.15.118:0/445036167 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00bb40 con 0x7f614c06b620 2024-08-18T13:50:58.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 -- 172.21.15.118:0/445036167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c06b620 msgr2=0x7f614c06b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:58.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 --2- 172.21.15.118:0/445036167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c06b620 0x7f614c06b9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f613c0098e0 tx=0x7f613c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 -- 172.21.15.118:0/445036167 shutdown_connections 2024-08-18T13:50:58.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 --2- 172.21.15.118:0/445036167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 --2- 172.21.15.118:0/445036167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c06b620 0x7f614c06b9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 -- 172.21.15.118:0/445036167 >> 172.21.15.118:0/445036167 conn(0x7f614c0c7c40 msgr2=0x7f614c0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:58.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 -- 172.21.15.118:0/445036167 shutdown_connections 2024-08-18T13:50:58.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 -- 172.21.15.118:0/445036167 wait complete. 2024-08-18T13:50:58.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 Processor -- start 2024-08-18T13:50:58.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.243+0000 7f615447e700 1 -- start start 2024-08-18T13:50:58.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f614c06b620 0x7f614c142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:58.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c0c2630 0x7f614c148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:58.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:58.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0c0190 con 0x7f614c06b620 2024-08-18T13:50:58.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c156ba0 con 0x7f614c0c2630 2024-08-18T13:50:58.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0be4b0 con 0x7f614c0c4030 2024-08-18T13:50:58.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f614c06b620 0x7f614c142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:58.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f6152a1b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:58.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f614c06b620 0x7f614c142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39472/0 (socket says 172.21.15.118:39472) 2024-08-18T13:50:58.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 -- 172.21.15.118:0/2287586252 learned_addr learned my addr 172.21.15.118:0/2287586252 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:50:58.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f6151a19700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c0c2630 0x7f614c148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:58.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 -- 172.21.15.118:0/2287586252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 msgr2=0x7f614c148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:58.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 -- 172.21.15.118:0/2287586252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c0c2630 msgr2=0x7f614c148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:58.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c0c2630 0x7f614c148540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 -- 172.21.15.118:0/2287586252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c009420 con 0x7f614c06b620 2024-08-18T13:50:58.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615221a700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f614c06b620 0x7f614c142ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f614c06d1d0 tx=0x7f613c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:58.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f61427fc700 1 -- 172.21.15.118:0/2287586252 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c01bc40 con 0x7f614c06b620 2024-08-18T13:50:58.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f61427fc700 1 -- 172.21.15.118:0/2287586252 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c01bdc0 con 0x7f614c06b620 2024-08-18T13:50:58.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f61427fc700 1 -- 172.21.15.118:0/2287586252 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c011670 con 0x7f614c06b620 2024-08-18T13:50:58.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c152c90 con 0x7f614c06b620 2024-08-18T13:50:58.255 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.247+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c0c6b90 con 0x7f614c06b620 2024-08-18T13:50:58.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.251+0000 7f61427fc700 1 -- 172.21.15.118:0/2287586252 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f613c011a20 con 0x7f614c06b620 2024-08-18T13:50:58.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.251+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614c0642f0 con 0x7f614c06b620 2024-08-18T13:50:58.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.251+0000 7f61427fc700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6130060860 0x7f6130062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:50:58.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.251+0000 7f61427fc700 1 -- 172.21.15.118:0/2287586252 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f613c07e700 con 0x7f614c06b620 2024-08-18T13:50:58.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.251+0000 7f6151a19700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6130060860 0x7f6130062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:50:58.256 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.251+0000 7f6151a19700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6130060860 0x7f6130062d10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f614c06ba60 tx=0x7f61440091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:50:58.265 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.263+0000 7f61427fc700 1 -- 172.21.15.118:0/2287586252 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f613c04e570 con 0x7f614c06b620 2024-08-18T13:50:58.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.463+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f614c143de0 con 0x7f614c06b620 2024-08-18T13:50:58.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f61427fc700 1 -- 172.21.15.118:0/2287586252 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5369 (secure 0 0 0) 0x7f613c051b90 con 0x7f614c06b620 2024-08-18T13:50:58.470 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:50:58.470 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:50.639803+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6130060860 msgr2=0x7f6130062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6130060860 0x7f6130062d10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f614c06ba60 tx=0x7f61440091a0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f614c06b620 msgr2=0x7f614c142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f614c06b620 0x7f614c142ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f614c06d1d0 tx=0x7f613c017000 comp rx=0 tx=0).stop 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 shutdown_connections 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6130060860 0x7f6130062d10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f614c0c4030 0x7f614c148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f614c0c2630 0x7f614c148540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 --2- 172.21.15.118:0/2287586252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f614c06b620 0x7f614c142ff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:50:58.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 >> 172.21.15.118:0/2287586252 conn(0x7f614c0c7c40 msgr2=0x7f614c1454c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:50:58.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 shutdown_connections 2024-08-18T13:50:58.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:50:58.467+0000 7f615447e700 1 -- 172.21.15.118:0/2287586252 wait complete. 2024-08-18T13:50:58.473 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 37 2024-08-18T13:50:58.483 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:51:00.484 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:00.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.595+0000 7f44d67ec700 1 Processor -- start 2024-08-18T13:51:00.600 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.595+0000 7f44d67ec700 1 -- start start 2024-08-18T13:51:00.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d67ec700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d014b840 0x7f44d0155580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:00.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d67ec700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f44d006b180 0x7f44d00c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:00.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d67ec700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d00c2de0 con 0x7f44d0069fc0 2024-08-18T13:51:00.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d67ec700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00c1d80 con 0x7f44d006b180 2024-08-18T13:51:00.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d67ec700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00c30c0 con 0x7f44d014b840 2024-08-18T13:51:00.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d4fe9700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f44d0069fc0 0x7f44d006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:56294/0 (socket says 172.21.15.118:56294) 2024-08-18T13:51:00.603 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d4fe9700 1 -- 172.21.15.118:0/2713545426 learned_addr learned my addr 172.21.15.118:0/2713545426 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:00.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d5feb700 1 --2- 172.21.15.118:0/2713545426 >> v2:172.21.15.178:3300/0 conn(0x7f44d006b180 0x7f44d00c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:00.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d57ea700 1 --2- 172.21.15.118:0/2713545426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d014b840 0x7f44d0155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:00.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44c7fff700 1 -- 172.21.15.118:0/2713545426 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961836243 0 0) 0x7f44d00c2de0 con 0x7f44d0069fc0 2024-08-18T13:51:00.604 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44c7fff700 1 -- 172.21.15.118:0/2713545426 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44bc003570 con 0x7f44d0069fc0 2024-08-18T13:51:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d57ea700 1 -- 172.21.15.118:0/2713545426 >> v2:172.21.15.178:3300/0 conn(0x7f44d006b180 msgr2=0x7f44d00c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d57ea700 1 --2- 172.21.15.118:0/2713545426 >> v2:172.21.15.178:3300/0 conn(0x7f44d006b180 0x7f44d00c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d57ea700 1 -- 172.21.15.118:0/2713545426 >> v1:172.21.15.118:6789/0 conn(0x7f44d0069fc0 legacy=0x7f44d006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d57ea700 1 -- 172.21.15.118:0/2713545426 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d006a490 con 0x7f44d014b840 2024-08-18T13:51:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d5feb700 1 --2- 172.21.15.118:0/2713545426 >> v2:172.21.15.178:3300/0 conn(0x7f44d006b180 0x7f44d00c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:51:00.605 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44d57ea700 1 --2- 172.21.15.118:0/2713545426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d014b840 0x7f44d0155580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0098e0 tx=0x7f44cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f9b084ff847102 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:00.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44c7fff700 1 -- 172.21.15.118:0/2713545426 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc011630 con 0x7f44d014b840 2024-08-18T13:51:00.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44c7fff700 1 -- 172.21.15.118:0/2713545426 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc011790 con 0x7f44d014b840 2024-08-18T13:51:00.606 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.599+0000 7f44c7fff700 1 -- 172.21.15.118:0/2713545426 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc011b30 con 0x7f44d014b840 2024-08-18T13:51:00.607 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 -- 172.21.15.118:0/2713545426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d014b840 msgr2=0x7f44d0155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 --2- 172.21.15.118:0/2713545426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d014b840 0x7f44d0155580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0098e0 tx=0x7f44cc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 -- 172.21.15.118:0/2713545426 shutdown_connections 2024-08-18T13:51:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 --2- 172.21.15.118:0/2713545426 >> v2:172.21.15.178:3300/0 conn(0x7f44d006b180 0x7f44d00c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 --2- 172.21.15.118:0/2713545426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d014b840 0x7f44d0155580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 -- 172.21.15.118:0/2713545426 >> 172.21.15.118:0/2713545426 conn(0x7f44d00c81e0 msgr2=0x7f44d00c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 -- 172.21.15.118:0/2713545426 shutdown_connections 2024-08-18T13:51:00.608 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 -- 172.21.15.118:0/2713545426 wait complete. 2024-08-18T13:51:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 Processor -- start 2024-08-18T13:51:00.609 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.603+0000 7f44d67ec700 1 -- start start 2024-08-18T13:51:00.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44d0069fc0 0x7f44d0166510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:00.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d006b180 0x7f44d0143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:00.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d014b840 0x7f44d01444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:00.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0151e20 con 0x7f44d0069fc0 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0165cb0 con 0x7f44d014b840 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0166fb0 con 0x7f44d006b180 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44d0069fc0 0x7f44d0166510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44d0069fc0 0x7f44d0166510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39482/0 (socket says 172.21.15.118:39482) 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 -- 172.21.15.118:0/4039776246 learned_addr learned my addr 172.21.15.118:0/4039776246 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d4fe9700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d006b180 0x7f44d0143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d5feb700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d014b840 0x7f44d01444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 -- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d006b180 msgr2=0x7f44d0143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d006b180 0x7f44d0143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 -- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d014b840 msgr2=0x7f44d01444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d014b840 0x7f44d01444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.613 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 -- 172.21.15.118:0/4039776246 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c800ab30 con 0x7f44d0069fc0 2024-08-18T13:51:00.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d57ea700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44d0069fc0 0x7f44d0166510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44d00c1ec0 tx=0x7f44cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:00.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44c57fa700 1 -- 172.21.15.118:0/4039776246 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc011900 con 0x7f44d0069fc0 2024-08-18T13:51:00.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc009420 con 0x7f44d0069fc0 2024-08-18T13:51:00.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44c57fa700 1 -- 172.21.15.118:0/4039776246 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc011a80 con 0x7f44d0069fc0 2024-08-18T13:51:00.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44c57fa700 1 -- 172.21.15.118:0/4039776246 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc025640 con 0x7f44d0069fc0 2024-08-18T13:51:00.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.607+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d0152030 con 0x7f44d0069fc0 2024-08-18T13:51:00.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.611+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d0166150 con 0x7f44d0069fc0 2024-08-18T13:51:00.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.611+0000 7f44c57fa700 1 -- 172.21.15.118:0/4039776246 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f44cc0259b0 con 0x7f44d0069fc0 2024-08-18T13:51:00.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.611+0000 7f44c57fa700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f44b4060890 0x7f44b4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:00.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.611+0000 7f44c57fa700 1 -- 172.21.15.118:0/4039776246 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f44cc07e7b0 con 0x7f44d0069fc0 2024-08-18T13:51:00.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.611+0000 7f44d4fe9700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f44b4060890 0x7f44b4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:00.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.611+0000 7f44d4fe9700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f44b4060890 0x7f44b4062d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f44d0142f90 tx=0x7f44c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:00.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.623+0000 7f44c57fa700 1 -- 172.21.15.118:0/4039776246 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f44cc04e640 con 0x7f44d0069fc0 2024-08-18T13:51:00.825 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.823+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44d00713a0 con 0x7f44d0069fc0 2024-08-18T13:51:00.827 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.823+0000 7f44c57fa700 1 -- 172.21.15.118:0/4039776246 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5373 (secure 0 0 0) 0x7f44cc051c60 con 0x7f44d0069fc0 2024-08-18T13:51:00.827 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:00.827 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:59.523648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:00.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.823+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f44b4060890 msgr2=0x7f44b4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:00.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.823+0000 7f44d67ec700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f44b4060890 0x7f44b4062d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f44d0142f90 tx=0x7f44c0009040 comp rx=0 tx=0).stop 2024-08-18T13:51:00.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.823+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44d0069fc0 msgr2=0x7f44d0166510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:00.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.823+0000 7f44d67ec700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44d0069fc0 0x7f44d0166510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44d00c1ec0 tx=0x7f44cc017000 comp rx=0 tx=0).stop 2024-08-18T13:51:00.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.823+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 shutdown_connections 2024-08-18T13:51:00.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.827+0000 7f44d67ec700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f44b4060890 0x7f44b4062d40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.830 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.827+0000 7f44d67ec700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44d014b840 0x7f44d01444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.827+0000 7f44d67ec700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44d006b180 0x7f44d0143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.827+0000 7f44d67ec700 1 --2- 172.21.15.118:0/4039776246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44d0069fc0 0x7f44d0166510 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:00.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.827+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 >> 172.21.15.118:0/4039776246 conn(0x7f44d00c81e0 msgr2=0x7f44d005de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:00.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.827+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 shutdown_connections 2024-08-18T13:51:00.831 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:00.827+0000 7f44d67ec700 1 -- 172.21.15.118:0/4039776246 wait complete. 2024-08-18T13:51:00.831 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 38 2024-08-18T13:51:00.846 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:51:02.848 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:02.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.959+0000 7f452c96b700 1 Processor -- start 2024-08-18T13:51:02.964 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.959+0000 7f452c96b700 1 -- start start 2024-08-18T13:51:02.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.959+0000 7f452c96b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280c6e90 0x7f4528155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:02.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.959+0000 7f452c96b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f45280bf0f0 0x7f45280bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:02.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.959+0000 7f452c96b700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280bffb0 con 0x7f45280bed20 2024-08-18T13:51:02.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.959+0000 7f452c96b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4528156140 con 0x7f45280bf0f0 2024-08-18T13:51:02.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.959+0000 7f452c96b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45280c0b50 con 0x7f45280c6e90 2024-08-18T13:51:02.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f45277fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f45280bed20 0x7f45280c0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:56306/0 (socket says 172.21.15.118:56306) 2024-08-18T13:51:02.966 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f45277fe700 1 -- 172.21.15.118:0/3741663910 learned_addr learned my addr 172.21.15.118:0/3741663910 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f4527fff700 1 --2- 172.21.15.118:0/3741663910 >> v2:172.21.15.178:3300/0 conn(0x7f45280bf0f0 0x7f45280bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f4526ffd700 1 --2- 172.21.15.118:0/3741663910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280c6e90 0x7f4528155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f45267fc700 1 -- 172.21.15.118:0/3741663910 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150230478 0 0) 0x7f45280bffb0 con 0x7f45280bed20 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f45267fc700 1 -- 172.21.15.118:0/3741663910 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4508003570 con 0x7f45280bed20 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f4526ffd700 1 -- 172.21.15.118:0/3741663910 >> v2:172.21.15.178:3300/0 conn(0x7f45280bf0f0 msgr2=0x7f45280bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f4526ffd700 1 --2- 172.21.15.118:0/3741663910 >> v2:172.21.15.178:3300/0 conn(0x7f45280bf0f0 0x7f45280bf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f4526ffd700 1 -- 172.21.15.118:0/3741663910 >> v1:172.21.15.118:6789/0 conn(0x7f45280bed20 legacy=0x7f45280c0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f4526ffd700 1 -- 172.21.15.118:0/3741663910 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280c7360 con 0x7f45280c6e90 2024-08-18T13:51:02.967 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f4526ffd700 1 --2- 172.21.15.118:0/3741663910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280c6e90 0x7f4528155c00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45100098e0 tx=0x7f45100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f23d6819722523d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f45267fc700 1 -- 172.21.15.118:0/3741663910 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510011630 con 0x7f45280c6e90 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f45267fc700 1 -- 172.21.15.118:0/3741663910 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4510011790 con 0x7f45280c6e90 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f45267fc700 1 -- 172.21.15.118:0/3741663910 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510011b30 con 0x7f45280c6e90 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 -- 172.21.15.118:0/3741663910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280c6e90 msgr2=0x7f4528155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 --2- 172.21.15.118:0/3741663910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280c6e90 0x7f4528155c00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45100098e0 tx=0x7f45100091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 -- 172.21.15.118:0/3741663910 shutdown_connections 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 --2- 172.21.15.118:0/3741663910 >> v2:172.21.15.178:3300/0 conn(0x7f45280bf0f0 0x7f45280bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 --2- 172.21.15.118:0/3741663910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280c6e90 0x7f4528155c00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 -- 172.21.15.118:0/3741663910 >> 172.21.15.118:0/3741663910 conn(0x7f45280ca520 msgr2=0x7f45280c65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 -- 172.21.15.118:0/3741663910 shutdown_connections 2024-08-18T13:51:02.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 -- 172.21.15.118:0/3741663910 wait complete. 2024-08-18T13:51:02.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 Processor -- start 2024-08-18T13:51:02.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.963+0000 7f452c96b700 1 -- start start 2024-08-18T13:51:02.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280bed20 0x7f4528150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:02.971 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45280bf0f0 0x7f4528149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45280c6e90 0x7f452814fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528165550 con 0x7f45280bf0f0 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528164c40 con 0x7f45280c6e90 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452814be40 con 0x7f45280bed20 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45280bf0f0 0x7f4528149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45280bf0f0 0x7f4528149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:39486/0 (socket says 172.21.15.118:39486) 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 -- 172.21.15.118:0/4282644694 learned_addr learned my addr 172.21.15.118:0/4282644694 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4527fff700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45280c6e90 0x7f452814fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f45277fe700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280bed20 0x7f4528150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 -- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280bed20 msgr2=0x7f4528150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280bed20 0x7f4528150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 -- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45280c6e90 msgr2=0x7f452814fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:02.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45280c6e90 0x7f452814fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:02.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 -- 172.21.15.118:0/4282644694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510009420 con 0x7f45280bf0f0 2024-08-18T13:51:02.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f4526ffd700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45280bf0f0 0x7f4528149d30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45280c0c90 tx=0x7f4510017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:02.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f451f7fe700 1 -- 172.21.15.118:0/4282644694 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510011900 con 0x7f45280bf0f0 2024-08-18T13:51:02.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f451f7fe700 1 -- 172.21.15.118:0/4282644694 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4510011aa0 con 0x7f45280bf0f0 2024-08-18T13:51:02.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f451f7fe700 1 -- 172.21.15.118:0/4282644694 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4510025670 con 0x7f45280bf0f0 2024-08-18T13:51:02.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452814c120 con 0x7f45280bf0f0 2024-08-18T13:51:02.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45281651e0 con 0x7f45280bf0f0 2024-08-18T13:51:02.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4528165700 con 0x7f45280bf0f0 2024-08-18T13:51:02.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.967+0000 7f451f7fe700 1 -- 172.21.15.118:0/4282644694 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4510023070 con 0x7f45280bf0f0 2024-08-18T13:51:02.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.971+0000 7f451f7fe700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f450c060860 0x7f450c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:02.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.971+0000 7f451f7fe700 1 -- 172.21.15.118:0/4282644694 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f451007e560 con 0x7f45280bf0f0 2024-08-18T13:51:02.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.971+0000 7f45277fe700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f450c060860 0x7f450c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:02.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.971+0000 7f45277fe700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f450c060860 0x7f450c062d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45180058d0 tx=0x7f4518009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:02.980 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:02.975+0000 7f451f7fe700 1 -- 172.21.15.118:0/4282644694 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4528165700 con 0x7f45280bf0f0 2024-08-18T13:51:03.181 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.175+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4528165700 con 0x7f45280bf0f0 2024-08-18T13:51:03.182 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f451f7fe700 1 -- 172.21.15.118:0/4282644694 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5373 (secure 0 0 0) 0x7f4528165700 con 0x7f45280bf0f0 2024-08-18T13:51:03.182 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:03.182 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:59.523648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:03.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f450c060860 msgr2=0x7f450c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:03.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f450c060860 0x7f450c062d10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45180058d0 tx=0x7f4518009040 comp rx=0 tx=0).stop 2024-08-18T13:51:03.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45280bf0f0 msgr2=0x7f4528149d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:03.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45280bf0f0 0x7f4528149d30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45280c0c90 tx=0x7f4510017000 comp rx=0 tx=0).stop 2024-08-18T13:51:03.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 shutdown_connections 2024-08-18T13:51:03.184 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f450c060860 0x7f450c062d10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:03.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45280c6e90 0x7f452814fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:03.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45280bf0f0 0x7f4528149d30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:03.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 --2- 172.21.15.118:0/4282644694 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45280bed20 0x7f4528150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:03.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 >> 172.21.15.118:0/4282644694 conn(0x7f45280ca520 msgr2=0x7f45280c5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:03.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 shutdown_connections 2024-08-18T13:51:03.185 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:03.179+0000 7f452c96b700 1 -- 172.21.15.118:0/4282644694 wait complete. 2024-08-18T13:51:03.186 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 38 2024-08-18T13:51:03.197 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 38, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 38, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5195": { "addr": "172.21.15.178:6837/3317475884", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6836", "nonce": 3317475884, "type": "v2" }, { "addr": "172.21.15.178:6837", "nonce": 3317475884, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5195, "incarnation": 37, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:reconnect", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:50:59.523648+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5195 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:51:05.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:05.311 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf75dca700 1 Processor -- start 2024-08-18T13:51:05.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf75dca700 1 -- start start 2024-08-18T13:51:05.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf75dca700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf701590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:05.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf75dca700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcf700c8d10 0x7fcf7014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:05.312 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf75dca700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf700ac990 con 0x7fcf700c7b10 2024-08-18T13:51:05.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf75dca700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf70159600 con 0x7fcf700c8d10 2024-08-18T13:51:05.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf75dca700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70156d20 con 0x7fcf700c7740 2024-08-18T13:51:05.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf755c9700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcf700c8d10 0x7fcf7014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:05.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf6ffff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fcf700c7b10 0x7fcf70156610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:51190/0 (socket says 172.21.15.118:51190) 2024-08-18T13:51:05.313 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf6ffff700 1 -- 172.21.15.118:0/3897232532 learned_addr learned my addr 172.21.15.118:0/3897232532 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf74dc8700 1 --2- 172.21.15.118:0/3897232532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf701590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf6f7fe700 1 -- 172.21.15.118:0/3897232532 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122639470 0 0) 0x7fcf700ac990 con 0x7fcf700c7b10 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.307+0000 7fcf6f7fe700 1 -- 172.21.15.118:0/3897232532 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf58003570 con 0x7fcf700c7b10 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf755c9700 1 -- 172.21.15.118:0/3897232532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 msgr2=0x7fcf701590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf755c9700 1 --2- 172.21.15.118:0/3897232532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf701590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf755c9700 1 -- 172.21.15.118:0/3897232532 >> v1:172.21.15.118:6789/0 conn(0x7fcf700c7b10 legacy=0x7fcf70156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf755c9700 1 -- 172.21.15.118:0/3897232532 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700ace20 con 0x7fcf700c8d10 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf755c9700 1 --2- 172.21.15.118:0/3897232532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf7014eea0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009910 tx=0x7fcf680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c73504e7baaea3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf6f7fe700 1 -- 172.21.15.118:0/3897232532 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800b6d0 con 0x7fcf700c8d10 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf6f7fe700 1 -- 172.21.15.118:0/3897232532 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6800b830 con 0x7fcf700c8d10 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf6f7fe700 1 -- 172.21.15.118:0/3897232532 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800bbf0 con 0x7fcf700c8d10 2024-08-18T13:51:05.315 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 -- 172.21.15.118:0/3897232532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 msgr2=0x7fcf7014eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 --2- 172.21.15.118:0/3897232532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf7014eea0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009910 tx=0x7fcf680091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 -- 172.21.15.118:0/3897232532 shutdown_connections 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 --2- 172.21.15.118:0/3897232532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf7014eea0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 --2- 172.21.15.118:0/3897232532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf701590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 -- 172.21.15.118:0/3897232532 >> 172.21.15.118:0/3897232532 conn(0x7fcf700cc610 msgr2=0x7fcf700c8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 -- 172.21.15.118:0/3897232532 shutdown_connections 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 -- 172.21.15.118:0/3897232532 wait complete. 2024-08-18T13:51:05.316 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 Processor -- start 2024-08-18T13:51:05.317 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.311+0000 7fcf75dca700 1 -- start start 2024-08-18T13:51:05.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf70067ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:05.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf700c7b10 0x7fcf70067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:05.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf701681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:05.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70165580 con 0x7fcf700c8d10 2024-08-18T13:51:05.319 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70164c70 con 0x7fcf700c7b10 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7014dd50 con 0x7fcf700c7740 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf701681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf701681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33224/0 (socket says 172.21.15.118:33224) 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 -- 172.21.15.118:0/1926513428 learned_addr learned my addr 172.21.15.118:0/1926513428 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf74dc8700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf70067ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf6ffff700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf700c7b10 0x7fcf70067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 -- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 msgr2=0x7fcf70067ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf70067ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 -- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf700c7b10 msgr2=0x7fcf70067570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf700c7b10 0x7fcf70067570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 -- 172.21.15.118:0/1926513428 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68009420 con 0x7fcf700c8d10 2024-08-18T13:51:05.320 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf755c9700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf701681f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009fb0 tx=0x7fcf68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:05.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf6cff9700 1 -- 172.21.15.118:0/1926513428 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6801f070 con 0x7fcf700c8d10 2024-08-18T13:51:05.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf6cff9700 1 -- 172.21.15.118:0/1926513428 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68011420 con 0x7fcf700c8d10 2024-08-18T13:51:05.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf6cff9700 1 -- 172.21.15.118:0/1926513428 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680117e0 con 0x7fcf700c8d10 2024-08-18T13:51:05.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7014e030 con 0x7fcf700c8d10 2024-08-18T13:51:05.321 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.315+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf70165210 con 0x7fcf700c8d10 2024-08-18T13:51:05.322 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.319+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf701656f0 con 0x7fcf700c8d10 2024-08-18T13:51:05.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.319+0000 7fcf6cff9700 1 -- 172.21.15.118:0/1926513428 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcf68011c10 con 0x7fcf700c8d10 2024-08-18T13:51:05.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.319+0000 7fcf6cff9700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf5c060860 0x7fcf5c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:05.323 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.319+0000 7fcf6cff9700 1 -- 172.21.15.118:0/1926513428 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fcf6807e700 con 0x7fcf700c8d10 2024-08-18T13:51:05.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.319+0000 7fcf74dc8700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf5c060860 0x7fcf5c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:05.324 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.319+0000 7fcf74dc8700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf5c060860 0x7fcf5c062d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006fd0 tx=0x7fcf64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:05.333 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.327+0000 7fcf6cff9700 1 -- 172.21.15.118:0/1926513428 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcf6804e590 con 0x7fcf700c8d10 2024-08-18T13:51:05.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf700713a0 con 0x7fcf700c8d10 2024-08-18T13:51:05.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf6cff9700 1 -- 172.21.15.118:0/1926513428 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5373 (secure 0 0 0) 0x7fcf68051bb0 con 0x7fcf700c8d10 2024-08-18T13:51:05.538 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:05.538 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:50:59.523648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf5c060860 msgr2=0x7fcf5c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf5c060860 0x7fcf5c062d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006fd0 tx=0x7fcf64008040 comp rx=0 tx=0).stop 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 msgr2=0x7fcf701681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf701681f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009fb0 tx=0x7fcf68017000 comp rx=0 tx=0).stop 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 shutdown_connections 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf5c060860 0x7fcf5c062d10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf700c8d10 0x7fcf701681f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf700c7b10 0x7fcf70067570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 --2- 172.21.15.118:0/1926513428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf700c7740 0x7fcf70067ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 >> 172.21.15.118:0/1926513428 conn(0x7fcf700cc610 msgr2=0x7fcf700cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:05.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 shutdown_connections 2024-08-18T13:51:05.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:05.535+0000 7fcf75dca700 1 -- 172.21.15.118:0/1926513428 wait complete. 2024-08-18T13:51:05.542 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 38 2024-08-18T13:51:05.553 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:51:07.554 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:07.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.663+0000 7ff6056ba700 1 Processor -- start 2024-08-18T13:51:07.668 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.663+0000 7ff6056ba700 1 -- start start 2024-08-18T13:51:07.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff60006c570 0x7ff60006c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:07.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6000c92f0 0x7ff600068690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:07.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000c9830 con 0x7ff60006ce80 2024-08-18T13:51:07.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff600153370 con 0x7ff60006c570 2024-08-18T13:51:07.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001524e0 con 0x7ff6000c92f0 2024-08-18T13:51:07.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5feffd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff60006c570 0x7ff60006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:07.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5f67fc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6000c92f0 0x7ff600068690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:07.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5ff7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff60006ce80 0x7ff600155a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:42326/0 (socket says 172.21.15.118:42326) 2024-08-18T13:51:07.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5ff7fe700 1 -- 172.21.15.118:0/1580163949 learned_addr learned my addr 172.21.15.118:0/1580163949 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5feffd700 1 -- 172.21.15.118:0/1580163949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6000c92f0 msgr2=0x7ff600068690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5feffd700 1 --2- 172.21.15.118:0/1580163949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6000c92f0 0x7ff600068690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5feffd700 1 -- 172.21.15.118:0/1580163949 >> v1:172.21.15.178:6789/0 conn(0x7ff60006ce80 legacy=0x7ff600155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5feffd700 1 -- 172.21.15.118:0/1580163949 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6001526f0 con 0x7ff60006c570 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5fe7fc700 1 -- 172.21.15.118:0/1580163949 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366975554 0 0) 0x7ff6000c9830 con 0x7ff60006ce80 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5feffd700 1 --2- 172.21.15.118:0/1580163949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 0x7ff60006c940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00098e0 tx=0x7ff5f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b28476816309239 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5fe7fc700 1 -- 172.21.15.118:0/1580163949 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000b640 con 0x7ff60006c570 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5fe7fc700 1 -- 172.21.15.118:0/1580163949 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f000b7a0 con 0x7ff60006c570 2024-08-18T13:51:07.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff5fe7fc700 1 -- 172.21.15.118:0/1580163949 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000bb40 con 0x7ff60006c570 2024-08-18T13:51:07.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- 172.21.15.118:0/1580163949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 msgr2=0x7ff60006c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:07.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 --2- 172.21.15.118:0/1580163949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 0x7ff60006c940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00098e0 tx=0x7ff5f00091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- 172.21.15.118:0/1580163949 shutdown_connections 2024-08-18T13:51:07.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 --2- 172.21.15.118:0/1580163949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff6000c92f0 0x7ff600068690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 --2- 172.21.15.118:0/1580163949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 0x7ff60006c940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- 172.21.15.118:0/1580163949 >> 172.21.15.118:0/1580163949 conn(0x7ff60006f6d0 msgr2=0x7ff6000c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:07.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- 172.21.15.118:0/1580163949 shutdown_connections 2024-08-18T13:51:07.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.667+0000 7ff6056ba700 1 -- 172.21.15.118:0/1580163949 wait complete. 2024-08-18T13:51:07.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 Processor -- start 2024-08-18T13:51:07.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 -- start start 2024-08-18T13:51:07.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 0x7ff600067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:07.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff60006ce80 0x7ff6000ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:07.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6000c92f0 0x7ff6000aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:07.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600151d20 con 0x7ff6000c92f0 2024-08-18T13:51:07.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60014a3a0 con 0x7ff60006c570 2024-08-18T13:51:07.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff6056ba700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60014aab0 con 0x7ff60006ce80 2024-08-18T13:51:07.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6000c92f0 0x7ff6000aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:07.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6000c92f0 0x7ff6000aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33236/0 (socket says 172.21.15.118:33236) 2024-08-18T13:51:07.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 -- 172.21.15.118:0/3255335667 learned_addr learned my addr 172.21.15.118:0/3255335667 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:07.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5feffd700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 0x7ff600067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:07.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5f67fc700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff60006ce80 0x7ff6000ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:07.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 -- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff60006ce80 msgr2=0x7ff6000ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:07.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff60006ce80 0x7ff6000ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 -- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 msgr2=0x7ff600067570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:07.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 0x7ff600067570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 -- 172.21.15.118:0/3255335667 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f0009420 con 0x7ff6000c92f0 2024-08-18T13:51:07.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.671+0000 7ff5ff7fe700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6000c92f0 0x7ff6000aded0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800ccc0 tx=0x7ff5f8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:07.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff5f77fe700 1 -- 172.21.15.118:0/3255335667 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800a8a0 con 0x7ff6000c92f0 2024-08-18T13:51:07.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff5f77fe700 1 -- 172.21.15.118:0/3255335667 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f800aa00 con 0x7ff6000c92f0 2024-08-18T13:51:07.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600158390 con 0x7ff6000c92f0 2024-08-18T13:51:07.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff5f77fe700 1 -- 172.21.15.118:0/3255335667 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800adc0 con 0x7ff6000c92f0 2024-08-18T13:51:07.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff600149a70 con 0x7ff6000c92f0 2024-08-18T13:51:07.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6000648f0 con 0x7ff6000c92f0 2024-08-18T13:51:07.679 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff5f77fe700 1 -- 172.21.15.118:0/3255335667 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff5f8007300 con 0x7ff6000c92f0 2024-08-18T13:51:07.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff5f77fe700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff5e0060860 0x7ff5e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:07.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff5f77fe700 1 -- 172.21.15.118:0/3255335667 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7ff5f807d510 con 0x7ff6000c92f0 2024-08-18T13:51:07.680 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.675+0000 7ff5feffd700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff5e0060860 0x7ff5e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:07.681 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.679+0000 7ff5feffd700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff5e0060860 0x7ff5e0062d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00093f0 tx=0x7ff5f0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:07.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.683+0000 7ff5f77fe700 1 -- 172.21.15.118:0/3255335667 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff5f804d3a0 con 0x7ff6000c92f0 2024-08-18T13:51:07.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.883+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6001684c0 con 0x7ff6000c92f0 2024-08-18T13:51:07.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff5f77fe700 1 -- 172.21.15.118:0/3255335667 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5370 (secure 0 0 0) 0x7ff5f80509c0 con 0x7ff6000c92f0 2024-08-18T13:51:07.890 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:07.890 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:05.553760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff5e0060860 msgr2=0x7ff5e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff5e0060860 0x7ff5e0062d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00093f0 tx=0x7ff5f0017000 comp rx=0 tx=0).stop 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6000c92f0 msgr2=0x7ff6000aded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6000c92f0 0x7ff6000aded0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800ccc0 tx=0x7ff5f8001a80 comp rx=0 tx=0).stop 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 shutdown_connections 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff5e0060860 0x7ff5e0062d10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff6000c92f0 0x7ff6000aded0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff60006ce80 0x7ff6000ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 --2- 172.21.15.118:0/3255335667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff60006c570 0x7ff600067570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:07.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 >> 172.21.15.118:0/3255335667 conn(0x7ff60006f6d0 msgr2=0x7ff6000c8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:07.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 shutdown_connections 2024-08-18T13:51:07.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:07.887+0000 7ff6056ba700 1 -- 172.21.15.118:0/3255335667 wait complete. 2024-08-18T13:51:07.894 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 39 2024-08-18T13:51:07.904 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:51:09.905 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:10.025 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994d1b7700 1 Processor -- start 2024-08-18T13:51:10.026 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994d1b7700 1 -- start start 2024-08-18T13:51:10.027 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994d1b7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994814bcd0 0x7f9948152410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994d1b7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9948151f80 con 0x7f99480bd580 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994d1b7700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9948151990 con 0x7f99480bea90 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994d1b7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994814c0a0 con 0x7f994814bcd0 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994659c700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f99480bd580 0x7f994806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:51198/0 (socket says 172.21.15.118:51198) 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f994659c700 1 -- 172.21.15.118:0/3040086593 learned_addr learned my addr 172.21.15.118:0/3040086593 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9946d9d700 1 --2- 172.21.15.118:0/3040086593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994814bcd0 0x7f9948152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9945d9b700 1 -- 172.21.15.118:0/3040086593 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620387859 0 0) 0x7f9948151f80 con 0x7f99480bd580 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9945d9b700 1 -- 172.21.15.118:0/3040086593 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9934003570 con 0x7f99480bd580 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9945d9b700 1 -- 172.21.15.118:0/3040086593 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881268943 0 0) 0x7f9948151990 con 0x7f99480bea90 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9946d9d700 1 -- 172.21.15.118:0/3040086593 >> v1:172.21.15.178:6789/0 conn(0x7f99480bea90 legacy=0x7f994806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9946d9d700 1 -- 172.21.15.118:0/3040086593 >> v1:172.21.15.118:6789/0 conn(0x7f99480bd580 legacy=0x7f994806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9946d9d700 1 -- 172.21.15.118:0/3040086593 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948152950 con 0x7f994814bcd0 2024-08-18T13:51:10.028 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9946d9d700 1 --2- 172.21.15.118:0/3040086593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994814bcd0 0x7f9948152410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9930009910 tx=0x7f99300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc837ff1c87fa71b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:10.029 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9945d9b700 1 -- 172.21.15.118:0/3040086593 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938001470 con 0x7f994814bcd0 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9945d9b700 1 -- 172.21.15.118:0/3040086593 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930011420 con 0x7f994814bcd0 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.023+0000 7f9945d9b700 1 -- 172.21.15.118:0/3040086593 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300117a0 con 0x7f994814bcd0 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- 172.21.15.118:0/3040086593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994814bcd0 msgr2=0x7f9948152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 --2- 172.21.15.118:0/3040086593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994814bcd0 0x7f9948152410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9930009910 tx=0x7f99300091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- 172.21.15.118:0/3040086593 shutdown_connections 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 --2- 172.21.15.118:0/3040086593 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f994814bcd0 0x7f9948152410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- 172.21.15.118:0/3040086593 >> 172.21.15.118:0/3040086593 conn(0x7f99480c7c40 msgr2=0x7f99480c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:10.030 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- 172.21.15.118:0/3040086593 shutdown_connections 2024-08-18T13:51:10.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- 172.21.15.118:0/3040086593 wait complete. 2024-08-18T13:51:10.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 Processor -- start 2024-08-18T13:51:10.031 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- start start 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99480bd580 0x7f99480beff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99480bea90 0x7f9948161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f994814bcd0 0x7f9948146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994806bf10 con 0x7f994814bcd0 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948161ac0 con 0x7f99480bd580 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994d1b7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99481614d0 con 0x7f99480bea90 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f994814bcd0 0x7f9948146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f994814bcd0 0x7f9948146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33240/0 (socket says 172.21.15.118:33240) 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994759e700 1 -- 172.21.15.118:0/56030715 learned_addr learned my addr 172.21.15.118:0/56030715 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f9946d9d700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99480bd580 0x7f99480beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.027+0000 7f994659c700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99480bea90 0x7f9948161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:10.034 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994759e700 1 -- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99480bea90 msgr2=0x7f9948161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994759e700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99480bea90 0x7f9948161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994759e700 1 -- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99480bd580 msgr2=0x7f99480beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994759e700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99480bd580 0x7f99480beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994759e700 1 -- 172.21.15.118:0/56030715 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930009420 con 0x7f994814bcd0 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994759e700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f994814bcd0 0x7f9948146d80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f994000c940 tx=0x7f99400016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f993effd700 1 -- 172.21.15.118:0/56030715 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940012630 con 0x7f994814bcd0 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f993effd700 1 -- 172.21.15.118:0/56030715 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9940012790 con 0x7f994814bcd0 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99480c1850 con 0x7f994814bcd0 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f993effd700 1 -- 172.21.15.118:0/56030715 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940012b50 con 0x7f994814bcd0 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99480c1e10 con 0x7f994814bcd0 2024-08-18T13:51:10.035 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99480642f0 con 0x7f994814bcd0 2024-08-18T13:51:10.036 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f993effd700 1 -- 172.21.15.118:0/56030715 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9940003030 con 0x7f994814bcd0 2024-08-18T13:51:10.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f993effd700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9928060860 0x7f9928062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:10.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f993effd700 1 -- 172.21.15.118:0/56030715 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f994007d110 con 0x7f994814bcd0 2024-08-18T13:51:10.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.031+0000 7f9946d9d700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9928060860 0x7f9928062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:10.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.035+0000 7f9946d9d700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9928060860 0x7f9928062d10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9930009860 tx=0x7f9930017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:10.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.043+0000 7f993effd700 1 -- 172.21.15.118:0/56030715 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f994004cfa0 con 0x7f994814bcd0 2024-08-18T13:51:10.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.243+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9948161c00 con 0x7f994814bcd0 2024-08-18T13:51:10.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f993effd700 1 -- 172.21.15.118:0/56030715 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f99400505c0 con 0x7f994814bcd0 2024-08-18T13:51:10.250 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:10.250 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:05.553760+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:10.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9928060860 msgr2=0x7f9928062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:10.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9928060860 0x7f9928062d10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9930009860 tx=0x7f9930017000 comp rx=0 tx=0).stop 2024-08-18T13:51:10.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f994814bcd0 msgr2=0x7f9948146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:10.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f994814bcd0 0x7f9948146d80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f994000c940 tx=0x7f99400016c0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 shutdown_connections 2024-08-18T13:51:10.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9928060860 0x7f9928062d10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.252 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f994814bcd0 0x7f9948146d80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99480bea90 0x7f9948161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 --2- 172.21.15.118:0/56030715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99480bd580 0x7f99480beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:10.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 >> 172.21.15.118:0/56030715 conn(0x7f99480c7c40 msgr2=0x7f9948067f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:10.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 shutdown_connections 2024-08-18T13:51:10.253 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:10.247+0000 7f994d1b7700 1 -- 172.21.15.118:0/56030715 wait complete. 2024-08-18T13:51:10.253 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 39 2024-08-18T13:51:10.264 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:51:12.265 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:12.382 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 Processor -- start 2024-08-18T13:51:12.383 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 -- start start 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f443006c6a0 0x7f4430155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f443006ca70 0x7f44300c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44300c90e0 0x7f44300c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44301563d0 con 0x7f443006ca70 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44300c1e80 con 0x7f443006c6a0 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.379+0000 7f4434c38700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c9690 con 0x7f44300c90e0 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f443006ca70 0x7f44300c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f443006ca70 0x7f44300c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33242/0 (socket says 172.21.15.118:33242) 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 -- 172.21.15.118:0/2384111869 learned_addr learned my addr 172.21.15.118:0/2384111869 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442f7fe700 1 --2- 172.21.15.118:0/2384111869 >> v2:172.21.15.178:3300/0 conn(0x7f443006c6a0 0x7f4430155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:12.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442ffff700 1 --2- 172.21.15.118:0/2384111869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44300c90e0 0x7f44300c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:12.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 -- 172.21.15.118:0/2384111869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44300c90e0 msgr2=0x7f44300c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:12.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 --2- 172.21.15.118:0/2384111869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44300c90e0 0x7f44300c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 -- 172.21.15.118:0/2384111869 >> v2:172.21.15.178:3300/0 conn(0x7f443006c6a0 msgr2=0x7f4430155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:12.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 --2- 172.21.15.118:0/2384111869 >> v2:172.21.15.178:3300/0 conn(0x7f443006c6a0 0x7f4430155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.387 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 -- 172.21.15.118:0/2384111869 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300c1550 con 0x7f443006ca70 2024-08-18T13:51:12.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442effd700 1 --2- 172.21.15.118:0/2384111869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006ca70 0x7f44300c2310 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f443006fd70 tx=0x7f4424009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70308e2a46ca931c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:12.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442e7fc700 1 -- 172.21.15.118:0/2384111869 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400b640 con 0x7f443006ca70 2024-08-18T13:51:12.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442e7fc700 1 -- 172.21.15.118:0/2384111869 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442400b7a0 con 0x7f443006ca70 2024-08-18T13:51:12.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f442e7fc700 1 -- 172.21.15.118:0/2384111869 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400bb60 con 0x7f443006ca70 2024-08-18T13:51:12.388 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 -- 172.21.15.118:0/2384111869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006ca70 msgr2=0x7f44300c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 --2- 172.21.15.118:0/2384111869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006ca70 0x7f44300c2310 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f443006fd70 tx=0x7f4424009250 comp rx=0 tx=0).stop 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 -- 172.21.15.118:0/2384111869 shutdown_connections 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 --2- 172.21.15.118:0/2384111869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44300c90e0 0x7f44300c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 --2- 172.21.15.118:0/2384111869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006ca70 0x7f44300c2310 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 --2- 172.21.15.118:0/2384111869 >> v2:172.21.15.178:3300/0 conn(0x7f443006c6a0 0x7f4430155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 -- 172.21.15.118:0/2384111869 >> 172.21.15.118:0/2384111869 conn(0x7f443006f4d0 msgr2=0x7f44300c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 -- 172.21.15.118:0/2384111869 shutdown_connections 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.383+0000 7f4434c38700 1 -- 172.21.15.118:0/2384111869 wait complete. 2024-08-18T13:51:12.389 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 Processor -- start 2024-08-18T13:51:12.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 -- start start 2024-08-18T13:51:12.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006c6a0 0x7f4430162a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:12.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f443006ca70 0x7f4430156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:12.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44300c90e0 0x7f44300c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:12.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300c4080 con 0x7f443006c6a0 2024-08-18T13:51:12.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4430141610 con 0x7f44300c90e0 2024-08-18T13:51:12.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4430142470 con 0x7f443006ca70 2024-08-18T13:51:12.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006c6a0 0x7f4430162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006c6a0 0x7f4430162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:33248/0 (socket says 172.21.15.118:33248) 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 -- 172.21.15.118:0/839915041 learned_addr learned my addr 172.21.15.118:0/839915041 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442ffff700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44300c90e0 0x7f44300c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 -- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f443006ca70 msgr2=0x7f4430156750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f443006ca70 0x7f4430156750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 -- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44300c90e0 msgr2=0x7f44300c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44300c90e0 0x7f44300c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 -- 172.21.15.118:0/839915041 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240094f0 con 0x7f443006c6a0 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442ffff700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44300c90e0 0x7f44300c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f442f7fe700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006c6a0 0x7f4430162a60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4420010e90 tx=0x7f44200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:12.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f44177fe700 1 -- 172.21.15.118:0/839915041 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442000edf0 con 0x7f443006c6a0 2024-08-18T13:51:12.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f44177fe700 1 -- 172.21.15.118:0/839915041 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420011450 con 0x7f443006c6a0 2024-08-18T13:51:12.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44300c5260 con 0x7f443006c6a0 2024-08-18T13:51:12.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f44177fe700 1 -- 172.21.15.118:0/839915041 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44200117f0 con 0x7f443006c6a0 2024-08-18T13:51:12.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.387+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4430141b40 con 0x7f443006c6a0 2024-08-18T13:51:12.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.391+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44300713a0 con 0x7f443006c6a0 2024-08-18T13:51:12.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.391+0000 7f44177fe700 1 -- 172.21.15.118:0/839915041 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4420011ba0 con 0x7f443006c6a0 2024-08-18T13:51:12.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.391+0000 7f44177fe700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f441c060890 0x7f441c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:12.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.391+0000 7f44177fe700 1 -- 172.21.15.118:0/839915041 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f442007f590 con 0x7f443006c6a0 2024-08-18T13:51:12.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.391+0000 7f442effd700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f441c060890 0x7f441c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:12.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.395+0000 7f442effd700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f441c060890 0x7f441c062d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f443006fd70 tx=0x7f4424017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:12.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.399+0000 7f44177fe700 1 -- 172.21.15.118:0/839915041 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f442004f420 con 0x7f443006c6a0 2024-08-18T13:51:12.611 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.607+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44300c0890 con 0x7f443006c6a0 2024-08-18T13:51:12.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.607+0000 7f44177fe700 1 -- 172.21.15.118:0/839915041 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5376 (secure 0 0 0) 0x7f4420052a40 con 0x7f443006c6a0 2024-08-18T13:51:12.612 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:12.612 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:11.572634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:clientreplay","state_seq":20,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:12.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f441c060890 msgr2=0x7f441c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f441c060890 0x7f441c062d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f443006fd70 tx=0x7f4424017000 comp rx=0 tx=0).stop 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006c6a0 msgr2=0x7f4430162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006c6a0 0x7f4430162a60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4420010e90 tx=0x7f44200091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 shutdown_connections 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f441c060890 0x7f441c062d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44300c90e0 0x7f44300c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f443006ca70 0x7f4430156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 --2- 172.21.15.118:0/839915041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f443006c6a0 0x7f4430162a60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 >> 172.21.15.118:0/839915041 conn(0x7f443006f4d0 msgr2=0x7f4430149c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 shutdown_connections 2024-08-18T13:51:12.615 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:12.611+0000 7f4434c38700 1 -- 172.21.15.118:0/839915041 wait complete. 2024-08-18T13:51:12.616 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 40 2024-08-18T13:51:12.626 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:51:14.316 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:51:14.319 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:51:14.627 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:14.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7ba32c700 1 Processor -- start 2024-08-18T13:51:14.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7ba32c700 1 -- start start 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7ba32c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fc7b40be5a0 0x7fc7b4151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7ba32c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b4152200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7ba32c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b406de00 con 0x7fc7b414b940 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7ba32c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b406fac0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7ba32c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4151870 con 0x7fc7b414bd10 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7b932a700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fc7b414b940 0x7fc7b406e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:54852/0 (socket says 172.21.15.118:54852) 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7b932a700 1 -- 172.21.15.118:0/1840000221 learned_addr learned my addr 172.21.15.118:0/1840000221 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7b8b29700 1 --2- 172.21.15.118:0/1840000221 >> v2:172.21.15.118:3300/0 conn(0x7fc7b40be5a0 0x7fc7b4151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:14.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7b9b2b700 1 --2- 172.21.15.118:0/1840000221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b4152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:14.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7abfff700 1 -- 172.21.15.118:0/1840000221 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629299077 0 0) 0x7fc7b406de00 con 0x7fc7b414b940 2024-08-18T13:51:14.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.739+0000 7fc7abfff700 1 -- 172.21.15.118:0/1840000221 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a0003570 con 0x7fc7b414b940 2024-08-18T13:51:14.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7b8b29700 1 -- 172.21.15.118:0/1840000221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 msgr2=0x7fc7b4152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7b8b29700 1 --2- 172.21.15.118:0/1840000221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b4152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7b8b29700 1 -- 172.21.15.118:0/1840000221 >> v1:172.21.15.118:6789/0 conn(0x7fc7b414b940 legacy=0x7fc7b406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7b8b29700 1 -- 172.21.15.118:0/1840000221 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b40c2da0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7b9b2b700 1 --2- 172.21.15.118:0/1840000221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b4152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7b8b29700 1 --2- 172.21.15.118:0/1840000221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b4151cc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc7b41519b0 tx=0x7fc7a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9408f3815f340b53 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7abfff700 1 -- 172.21.15.118:0/1840000221 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4011630 con 0x7fc7b40be5a0 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7abfff700 1 -- 172.21.15.118:0/1840000221 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a4011790 con 0x7fc7b40be5a0 2024-08-18T13:51:14.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7abfff700 1 -- 172.21.15.118:0/1840000221 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4011b10 con 0x7fc7b40be5a0 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 -- 172.21.15.118:0/1840000221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 msgr2=0x7fc7b4151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/1840000221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b4151cc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc7b41519b0 tx=0x7fc7a40091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 -- 172.21.15.118:0/1840000221 shutdown_connections 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/1840000221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b4152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/1840000221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b4151cc0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 -- 172.21.15.118:0/1840000221 >> 172.21.15.118:0/1840000221 conn(0x7fc7b406c020 msgr2=0x7fc7b40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 -- 172.21.15.118:0/1840000221 shutdown_connections 2024-08-18T13:51:14.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.743+0000 7fc7ba32c700 1 -- 172.21.15.118:0/1840000221 wait complete. 2024-08-18T13:51:14.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 Processor -- start 2024-08-18T13:51:14.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 -- start start 2024-08-18T13:51:14.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b41477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b414b940 0x7fc7b40c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b40c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b40bfc50 con 0x7fc7b40be5a0 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b413da70 con 0x7fc7b414b940 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7ba32c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4145bb0 con 0x7fc7b414bd10 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b932a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b41477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b932a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b41477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:35532/0 (socket says 172.21.15.118:35532) 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b932a700 1 -- 172.21.15.118:0/3303689046 learned_addr learned my addr 172.21.15.118:0/3303689046 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b9b2b700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b40c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:14.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b8b29700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b414b940 0x7fc7b40c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b932a700 1 -- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 msgr2=0x7fc7b40c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b932a700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b40c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b932a700 1 -- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b414b940 msgr2=0x7fc7b40c00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.747+0000 7fc7b932a700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b414b940 0x7fc7b40c00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7b932a700 1 -- 172.21.15.118:0/3303689046 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b0009420 con 0x7fc7b40be5a0 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7b8b29700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b414b940 0x7fc7b40c00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7b932a700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b41477e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00cd00 tx=0x7fc7ac001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7a97fa700 1 -- 172.21.15.118:0/3303689046 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00aba0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7a97fa700 1 -- 172.21.15.118:0/3303689046 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac00ad00 con 0x7fc7b40be5a0 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a4009420 con 0x7fc7b40be5a0 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7a97fa700 1 -- 172.21.15.118:0/3303689046 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac007300 con 0x7fc7b40be5a0 2024-08-18T13:51:14.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4146170 con 0x7fc7b40be5a0 2024-08-18T13:51:14.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b40713a0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7a97fa700 1 -- 172.21.15.118:0/3303689046 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc7ac0076b0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.751+0000 7fc7a97fa700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc798060860 0x7fc798062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:14.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.755+0000 7fc7b8b29700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc798060860 0x7fc798062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:14.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.755+0000 7fc7a97fa700 1 -- 172.21.15.118:0/3303689046 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fc7ac07d5d0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.757 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.755+0000 7fc7b8b29700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc798060860 0x7fc798062d10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc7b41519b0 tx=0x7fc7a4016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:14.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.763+0000 7fc7a97fa700 1 -- 172.21.15.118:0/3303689046 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc7ac04d5a0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.969 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.963+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7b40bee50 con 0x7fc7b40be5a0 2024-08-18T13:51:14.970 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7a97fa700 1 -- 172.21.15.118:0/3303689046 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5376 (secure 0 0 0) 0x7fc7ac050bc0 con 0x7fc7b40be5a0 2024-08-18T13:51:14.970 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:14.970 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:11.572634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:clientreplay","state_seq":20,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:14.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc798060860 msgr2=0x7fc798062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:14.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc798060860 0x7fc798062d10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc7b41519b0 tx=0x7fc7a4016000 comp rx=0 tx=0).stop 2024-08-18T13:51:14.972 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 msgr2=0x7fc7b41477e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b41477e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00cd00 tx=0x7fc7ac001b00 comp rx=0 tx=0).stop 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 shutdown_connections 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc798060860 0x7fc798062d10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7b414bd10 0x7fc7b40c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7b414b940 0x7fc7b40c00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 --2- 172.21.15.118:0/3303689046 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7b40be5a0 0x7fc7b41477e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 >> 172.21.15.118:0/3303689046 conn(0x7fc7b406c020 msgr2=0x7fc7b4151bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 shutdown_connections 2024-08-18T13:51:14.973 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:14.967+0000 7fc7ba32c700 1 -- 172.21.15.118:0/3303689046 wait complete. 2024-08-18T13:51:14.974 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 40 2024-08-18T13:51:14.984 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 40, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-08-18T13:39:34.735047+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 40, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5195": { "addr": "172.21.15.178:6837/3317475884", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6836", "nonce": 3317475884, "type": "v2" }, { "addr": "172.21.15.178:6837", "nonce": 3317475884, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5195, "incarnation": 37, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:clientreplay", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 2, "modified": "2024-08-18T13:51:11.572634+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5195 } } } ], "standbys": [ { "addr": "172.21.15.118:6835/3872383872", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6834", "nonce": 3872383872, "type": "v2" }, { "addr": "172.21.15.118:6835", "nonce": 3872383872, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4275, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6837/1680290813", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6836", "nonce": 1680290813, "type": "v2" }, { "addr": "172.21.15.118:6837", "nonce": 1680290813, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4284, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.118:6839/2444565967", "addrs": { "addrvec": [ { "addr": "172.21.15.118:6838", "nonce": 2444565967, "type": "v2" }, { "addr": "172.21.15.118:6839", "nonce": 2444565967, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4296, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.178:6834/4030958675", "addrs": { "addrvec": [ { "addr": "172.21.15.178:6833", "nonce": 4030958675, "type": "v2" }, { "addr": "172.21.15.178:6834", "nonce": 4030958675, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4303, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-08-18T13:51:16.985 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:17.092 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.087+0000 7f0793fff700 1 Processor -- start 2024-08-18T13:51:17.093 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.087+0000 7f0793fff700 1 -- start start 2024-08-18T13:51:17.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:17.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940bffb0 con 0x7f07940c6e90 2024-08-18T13:51:17.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940c0b50 con 0x7f07940bed20 2024-08-18T13:51:17.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07941560a0 con 0x7f0794155c00 2024-08-18T13:51:17.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f07927fc700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f07940c6e90 0x7f07940bf0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:54856/0 (socket says 172.21.15.118:54856) 2024-08-18T13:51:17.094 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f07927fc700 1 -- 172.21.15.118:0/4220696698 learned_addr learned my addr 172.21.15.118:0/4220696698 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:17.095 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f07937fe700 1 --2- 172.21.15.118:0/4220696698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0791ffb700 1 -- 172.21.15.118:0/4220696698 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734064889 0 0) 0x7f07940c0b50 con 0x7f07940bed20 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0791ffb700 1 -- 172.21.15.118:0/4220696698 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f077c003570 con 0x7f07940bed20 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0791ffb700 1 -- 172.21.15.118:0/4220696698 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151508972 0 0) 0x7f07940bffb0 con 0x7f07940c6e90 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0791ffb700 1 -- 172.21.15.118:0/4220696698 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07940c0b50 con 0x7f07940c6e90 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f07937fe700 1 -- 172.21.15.118:0/4220696698 >> v1:172.21.15.178:6789/0 conn(0x7f07940bed20 legacy=0x7f07940c0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f07937fe700 1 -- 172.21.15.118:0/4220696698 >> v1:172.21.15.118:6789/0 conn(0x7f07940c6e90 legacy=0x7f07940bf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f07937fe700 1 -- 172.21.15.118:0/4220696698 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940c7360 con 0x7f0794155c00 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f07937fe700 1 --2- 172.21.15.118:0/4220696698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079415fd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f078c009e90 tx=0x7f078c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c47c7bae7f4cbb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0791ffb700 1 -- 172.21.15.118:0/4220696698 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00e070 con 0x7f0794155c00 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0791ffb700 1 -- 172.21.15.118:0/4220696698 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00bb80 con 0x7f0794155c00 2024-08-18T13:51:17.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0791ffb700 1 -- 172.21.15.118:0/4220696698 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c012430 con 0x7f0794155c00 2024-08-18T13:51:17.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- 172.21.15.118:0/4220696698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 msgr2=0x7f079415fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:17.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 --2- 172.21.15.118:0/4220696698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079415fd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f078c009e90 tx=0x7f078c009780 comp rx=0 tx=0).stop 2024-08-18T13:51:17.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- 172.21.15.118:0/4220696698 shutdown_connections 2024-08-18T13:51:17.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 --2- 172.21.15.118:0/4220696698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079415fd80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:17.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- 172.21.15.118:0/4220696698 >> 172.21.15.118:0/4220696698 conn(0x7f07940ca520 msgr2=0x7f07940c65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:17.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- 172.21.15.118:0/4220696698 shutdown_connections 2024-08-18T13:51:17.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.091+0000 7f0793fff700 1 -- 172.21.15.118:0/4220696698 wait complete. 2024-08-18T13:51:17.098 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 Processor -- start 2024-08-18T13:51:17.099 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 -- start start 2024-08-18T13:51:17.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07940bed20 0x7f07940c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:17.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07940c6e90 0x7f079414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:17.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079414af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:17.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f07927fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07940c6e90 0x7f079414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:17.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f07927fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07940c6e90 0x7f079414a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:35534/0 (socket says 172.21.15.118:35534) 2024-08-18T13:51:17.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f07927fc700 1 -- 172.21.15.118:0/3447992181 learned_addr learned my addr 172.21.15.118:0/3447992181 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:17.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0792ffd700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07940bed20 0x7f07940c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:17.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f07937fe700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079414af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:17.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0794149600 con 0x7f07940c6e90 2024-08-18T13:51:17.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c1a70 con 0x7f07940bed20 2024-08-18T13:51:17.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.095+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0794149360 con 0x7f0794155c00 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f07927fc700 1 -- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 msgr2=0x7f079414af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f07927fc700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079414af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f07927fc700 1 -- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07940bed20 msgr2=0x7f07940c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f07927fc700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07940bed20 0x7f07940c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f07927fc700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c009a20 con 0x7f07940c6e90 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f07927fc700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07940c6e90 0x7f079414a9d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f078800c590 tx=0x7f0788007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f077affd700 1 -- 172.21.15.118:0/3447992181 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788005b30 con 0x7f07940c6e90 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f077affd700 1 -- 172.21.15.118:0/3447992181 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788005c90 con 0x7f07940c6e90 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07940c1cf0 con 0x7f07940c6e90 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079414a1d0 con 0x7f07940c6e90 2024-08-18T13:51:17.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f077affd700 1 -- 172.21.15.118:0/3447992181 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880072e0 con 0x7f07940c6e90 2024-08-18T13:51:17.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07740052b0 con 0x7f07940c6e90 2024-08-18T13:51:17.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.099+0000 7f077affd700 1 -- 172.21.15.118:0/3447992181 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0788007690 con 0x7f07940c6e90 2024-08-18T13:51:17.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.103+0000 7f077affd700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0780060860 0x7f0780062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:17.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.103+0000 7f077affd700 1 -- 172.21.15.118:0/3447992181 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f078807c4a0 con 0x7f07940c6e90 2024-08-18T13:51:17.105 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.103+0000 7f0792ffd700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0780060860 0x7f0780062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:17.106 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.103+0000 7f0792ffd700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0780060860 0x7f0780062d10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0784007fd0 tx=0x7f0784007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:17.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.107+0000 7f077affd700 1 -- 172.21.15.118:0/3447992181 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f078804c370 con 0x7f07940c6e90 2024-08-18T13:51:17.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.391+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0774005560 con 0x7f07940c6e90 2024-08-18T13:51:17.397 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.395+0000 7f077affd700 1 -- 172.21.15.118:0/3447992181 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5376 (secure 0 0 0) 0x7f078804f990 con 0x7f07940c6e90 2024-08-18T13:51:17.398 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:17.398 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:11.572634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:clientreplay","state_seq":20,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0780060860 msgr2=0x7f0780062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0780060860 0x7f0780062d10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0784007fd0 tx=0x7f0784007d50 comp rx=0 tx=0).stop 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07940c6e90 msgr2=0x7f079414a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07940c6e90 0x7f079414a9d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f078800c590 tx=0x7f0788007d90 comp rx=0 tx=0).stop 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 shutdown_connections 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0780060860 0x7f0780062d10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0794155c00 0x7f079414af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f07940c6e90 0x7f079414a9d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 --2- 172.21.15.118:0/3447992181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07940bed20 0x7f07940c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 >> 172.21.15.118:0/3447992181 conn(0x7f07940ca520 msgr2=0x7f07941547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 shutdown_connections 2024-08-18T13:51:17.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:17.399+0000 7f0793fff700 1 -- 172.21.15.118:0/3447992181 wait complete. 2024-08-18T13:51:17.403 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 40 2024-08-18T13:51:17.415 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-08-18T13:51:19.416 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:19.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.523+0000 7fcf4ae66700 1 Processor -- start 2024-08-18T13:51:19.528 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.523+0000 7fcf4ae66700 1 -- start start 2024-08-18T13:51:19.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf440c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf440abea0 0x7fcf440c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.529 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf440c1c80 con 0x7fcf4414f410 2024-08-18T13:51:19.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440c18e0 con 0x7fcf440abea0 2024-08-18T13:51:19.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440c09f0 con 0x7fcf4414f7e0 2024-08-18T13:51:19.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf49e64700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fcf4414f410 0x7fcf440ab790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:54870/0 (socket says 172.21.15.118:54870) 2024-08-18T13:51:19.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf49e64700 1 -- 172.21.15.118:0/2657116936 learned_addr learned my addr 172.21.15.118:0/2657116936 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:19.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4a665700 1 --2- 172.21.15.118:0/2657116936 >> v2:172.21.15.178:3300/0 conn(0x7fcf440abea0 0x7fcf440c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf49663700 1 --2- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf440c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf48e62700 1 -- 172.21.15.118:0/2657116936 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914266940 0 0) 0x7fcf440c1c80 con 0x7fcf4414f410 2024-08-18T13:51:19.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf48e62700 1 -- 172.21.15.118:0/2657116936 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf30003570 con 0x7fcf4414f410 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4a665700 1 -- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 msgr2=0x7fcf440c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4a665700 1 --2- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf440c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4a665700 1 -- 172.21.15.118:0/2657116936 >> v1:172.21.15.118:6789/0 conn(0x7fcf4414f410 legacy=0x7fcf440ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4a665700 1 -- 172.21.15.118:0/2657116936 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440c0cd0 con 0x7fcf440abea0 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4a665700 1 --2- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf440abea0 0x7fcf440c7a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009e60 tx=0x7fcf3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83de345ad307c477 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf48e62700 1 -- 172.21.15.118:0/2657116936 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00e070 con 0x7fcf440abea0 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf48e62700 1 -- 172.21.15.118:0/2657116936 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00bb40 con 0x7fcf440abea0 2024-08-18T13:51:19.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf48e62700 1 -- 172.21.15.118:0/2657116936 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c012430 con 0x7fcf440abea0 2024-08-18T13:51:19.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf440abea0 msgr2=0x7fcf440c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf440abea0 0x7fcf440c7a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009e60 tx=0x7fcf3c009780 comp rx=0 tx=0).stop 2024-08-18T13:51:19.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- 172.21.15.118:0/2657116936 shutdown_connections 2024-08-18T13:51:19.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf440abea0 0x7fcf440c7a70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/2657116936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf440c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- 172.21.15.118:0/2657116936 >> 172.21.15.118:0/2657116936 conn(0x7fcf440c8620 msgr2=0x7fcf440c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:19.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- 172.21.15.118:0/2657116936 shutdown_connections 2024-08-18T13:51:19.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.527+0000 7fcf4ae66700 1 -- 172.21.15.118:0/2657116936 wait complete. 2024-08-18T13:51:19.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.531+0000 7fcf4ae66700 1 Processor -- start 2024-08-18T13:51:19.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.531+0000 7fcf4ae66700 1 -- start start 2024-08-18T13:51:19.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf440abea0 0x7fcf44159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf4414f410 0x7fcf4414ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf4414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44156490 con 0x7fcf440abea0 2024-08-18T13:51:19.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4415a810 con 0x7fcf4414f410 2024-08-18T13:51:19.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4415a530 con 0x7fcf4414f7e0 2024-08-18T13:51:19.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf440abea0 0x7fcf44159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf440abea0 0x7fcf44159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:35546/0 (socket says 172.21.15.118:35546) 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 -- 172.21.15.118:0/530261451 learned_addr learned my addr 172.21.15.118:0/530261451 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49663700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf4414f410 0x7fcf4414ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4a665700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf4414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 -- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 msgr2=0x7fcf4414b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf4414b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 -- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf4414f410 msgr2=0x7fcf4414ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf4414f410 0x7fcf4414ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 -- 172.21.15.118:0/530261451 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c009a20 con 0x7fcf440abea0 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf49e64700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf440abea0 0x7fcf44159a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcf400011e0 tx=0x7fcf40007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf39ffb700 1 -- 172.21.15.118:0/530261451 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40005b30 con 0x7fcf440abea0 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf39ffb700 1 -- 172.21.15.118:0/530261451 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf40005c90 con 0x7fcf440abea0 2024-08-18T13:51:19.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf39ffb700 1 -- 172.21.15.118:0/530261451 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40007300 con 0x7fcf440abea0 2024-08-18T13:51:19.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf44149730 con 0x7fcf440abea0 2024-08-18T13:51:19.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4414ba20 con 0x7fcf440abea0 2024-08-18T13:51:19.541 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.535+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf44149590 con 0x7fcf440abea0 2024-08-18T13:51:19.542 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.539+0000 7fcf39ffb700 1 -- 172.21.15.118:0/530261451 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcf40007670 con 0x7fcf440abea0 2024-08-18T13:51:19.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.539+0000 7fcf39ffb700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf28060890 0x7fcf28062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.539+0000 7fcf39ffb700 1 -- 172.21.15.118:0/530261451 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fcf4007c530 con 0x7fcf440abea0 2024-08-18T13:51:19.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.539+0000 7fcf49663700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf28060890 0x7fcf28062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.539+0000 7fcf49663700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf28060890 0x7fcf28062d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf44156300 tx=0x7fcf340091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:19.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.547+0000 7fcf39ffb700 1 -- 172.21.15.118:0/530261451 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcf44149590 con 0x7fcf440abea0 2024-08-18T13:51:19.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.747+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf44149590 con 0x7fcf440abea0 2024-08-18T13:51:19.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.747+0000 7fcf39ffb700 1 -- 172.21.15.118:0/530261451 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5370 (secure 0 0 0) 0x7fcf44149590 con 0x7fcf440abea0 2024-08-18T13:51:19.752 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:19.752 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:18.658279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":22,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:19.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf28060890 msgr2=0x7fcf28062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf28060890 0x7fcf28062d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf44156300 tx=0x7fcf340091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf440abea0 msgr2=0x7fcf44159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf440abea0 0x7fcf44159a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcf400011e0 tx=0x7fcf40007df0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.754 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 shutdown_connections 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcf28060890 0x7fcf28062d40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf4414f7e0 0x7fcf4414b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf4414f410 0x7fcf4414ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 --2- 172.21.15.118:0/530261451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf440abea0 0x7fcf44159a90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 >> 172.21.15.118:0/530261451 conn(0x7fcf440c8620 msgr2=0x7fcf440c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 shutdown_connections 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.751+0000 7fcf4ae66700 1 -- 172.21.15.118:0/530261451 wait complete. 2024-08-18T13:51:19.755 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 41 2024-08-18T13:51:19.766 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=0, replacing gid=5464 2024-08-18T13:51:19.766 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.f 2024-08-18T13:51:19.766 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-08-18T13:51:19.766 INFO:tasks.ceph.mds.f:Restarting daemon 2024-08-18T13:51:19.767 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-08-18T13:51:19.769 INFO:tasks.ceph.mds.f:Started 2024-08-18T13:51:19.769 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:19.848 INFO:tasks.ceph.mds.f.smithi178.stdout:starting mds.f at 2024-08-18T13:51:19.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941ffff700 1 Processor -- start 2024-08-18T13:51:19.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941ffff700 1 -- start start 2024-08-18T13:51:19.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941ffff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f94200abea0 0x7f94200c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941ffff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94200c1c80 con 0x7f942014f410 2024-08-18T13:51:19.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941ffff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94200c18e0 con 0x7f942014f7e0 2024-08-18T13:51:19.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941ffff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200c8180 con 0x7f94200abea0 2024-08-18T13:51:19.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941effd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f942014f410 0x7f94200ab790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:54878/0 (socket says 172.21.15.118:54878) 2024-08-18T13:51:19.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941effd700 1 -- 172.21.15.118:0/3200939111 learned_addr learned my addr 172.21.15.118:0/3200939111 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:19.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941f7fe700 1 --2- 172.21.15.118:0/3200939111 >> v2:172.21.15.178:3300/0 conn(0x7f94200abea0 0x7f94200c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941dffb700 1 -- 172.21.15.118:0/3200939111 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767076818 0 0) 0x7f94200c1c80 con 0x7f942014f410 2024-08-18T13:51:19.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941dffb700 1 -- 172.21.15.118:0/3200939111 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f940c003570 con 0x7f942014f410 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941dffb700 1 -- 172.21.15.118:0/3200939111 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970037353 0 0) 0x7f94200c18e0 con 0x7f942014f7e0 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941dffb700 1 -- 172.21.15.118:0/3200939111 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94200c1c80 con 0x7f942014f7e0 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941f7fe700 1 -- 172.21.15.118:0/3200939111 >> v1:172.21.15.178:6789/0 conn(0x7f942014f7e0 legacy=0x7f94200c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941f7fe700 1 -- 172.21.15.118:0/3200939111 >> v1:172.21.15.118:6789/0 conn(0x7f942014f410 legacy=0x7f94200ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941f7fe700 1 -- 172.21.15.118:0/3200939111 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200c09f0 con 0x7f94200abea0 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.875+0000 7f941f7fe700 1 --2- 172.21.15.118:0/3200939111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94200abea0 0x7f94200c7c40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94180098e0 tx=0x7f94180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10a7c166a8ad954 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941dffb700 1 -- 172.21.15.118:0/3200939111 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418011630 con 0x7f94200abea0 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941dffb700 1 -- 172.21.15.118:0/3200939111 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9418011790 con 0x7f94200abea0 2024-08-18T13:51:19.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941dffb700 1 -- 172.21.15.118:0/3200939111 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418011b30 con 0x7f94200abea0 2024-08-18T13:51:19.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 -- 172.21.15.118:0/3200939111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94200abea0 msgr2=0x7f94200c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 --2- 172.21.15.118:0/3200939111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94200abea0 0x7f94200c7c40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f94180098e0 tx=0x7f94180091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 -- 172.21.15.118:0/3200939111 shutdown_connections 2024-08-18T13:51:19.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 --2- 172.21.15.118:0/3200939111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94200abea0 0x7f94200c7c40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 -- 172.21.15.118:0/3200939111 >> 172.21.15.118:0/3200939111 conn(0x7f94200c8620 msgr2=0x7f94200c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:19.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 -- 172.21.15.118:0/3200939111 shutdown_connections 2024-08-18T13:51:19.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 -- 172.21.15.118:0/3200939111 wait complete. 2024-08-18T13:51:19.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 Processor -- start 2024-08-18T13:51:19.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.879+0000 7f941ffff700 1 -- start start 2024-08-18T13:51:19.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94200abea0 0x7f94200aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942014f410 0x7f942014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942014f7e0 0x7f942014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200ab990 con 0x7f94200abea0 2024-08-18T13:51:19.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420154620 con 0x7f942014f410 2024-08-18T13:51:19.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9420154bc0 con 0x7f942014f7e0 2024-08-18T13:51:19.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94200abea0 0x7f94200aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94200abea0 0x7f94200aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:35560/0 (socket says 172.21.15.118:35560) 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 -- 172.21.15.118:0/1143507661 learned_addr learned my addr 172.21.15.118:0/1143507661 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941f7fe700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942014f7e0 0x7f942014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941e7fc700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942014f410 0x7f942014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 -- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942014f7e0 msgr2=0x7f942014cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942014f7e0 0x7f942014cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 -- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942014f410 msgr2=0x7f942014c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942014f410 0x7f942014c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 -- 172.21.15.118:0/1143507661 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9418009420 con 0x7f94200abea0 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941e7fc700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942014f410 0x7f942014c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941effd700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94200abea0 0x7f94200aca70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f940800c790 tx=0x7f9408007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:19.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f9416ffd700 1 -- 172.21.15.118:0/1143507661 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408005b30 con 0x7f94200abea0 2024-08-18T13:51:19.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f9416ffd700 1 -- 172.21.15.118:0/1143507661 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9408005c90 con 0x7f94200abea0 2024-08-18T13:51:19.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94200abbc0 con 0x7f94200abea0 2024-08-18T13:51:19.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f9416ffd700 1 -- 172.21.15.118:0/1143507661 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408007300 con 0x7f94200abea0 2024-08-18T13:51:19.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9420154850 con 0x7f94200abea0 2024-08-18T13:51:19.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94200713a0 con 0x7f94200abea0 2024-08-18T13:51:19.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.883+0000 7f9416ffd700 1 -- 172.21.15.118:0/1143507661 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f940801b400 con 0x7f94200abea0 2024-08-18T13:51:19.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.887+0000 7f9416ffd700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9400060860 0x7f9400062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:19.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.887+0000 7f9416ffd700 1 -- 172.21.15.118:0/1143507661 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f940807cca0 con 0x7f94200abea0 2024-08-18T13:51:19.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.887+0000 7f941e7fc700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9400060860 0x7f9400062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:19.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.887+0000 7f941e7fc700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9400060860 0x7f9400062d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f942006a6c0 tx=0x7f9410009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:19.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:19.891+0000 7f9416ffd700 1 -- 172.21.15.118:0/1143507661 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9408050640 con 0x7f94200abea0 2024-08-18T13:51:20.096 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.091+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f942014de10 con 0x7f94200abea0 2024-08-18T13:51:20.097 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f9416ffd700 1 -- 172.21.15.118:0/1143507661 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5370 (secure 0 0 0) 0x7f94080501d0 con 0x7f94200abea0 2024-08-18T13:51:20.097 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:20.097 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:18.658279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":22,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9400060860 msgr2=0x7f9400062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9400060860 0x7f9400062d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f942006a6c0 tx=0x7f9410009040 comp rx=0 tx=0).stop 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94200abea0 msgr2=0x7f94200aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94200abea0 0x7f94200aca70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f940800c790 tx=0x7f9408007ea0 comp rx=0 tx=0).stop 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 shutdown_connections 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9400060860 0x7f9400062d10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f942014f7e0 0x7f942014cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f942014f410 0x7f942014c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 --2- 172.21.15.118:0/1143507661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94200abea0 0x7f94200aca70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:20.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 >> 172.21.15.118:0/1143507661 conn(0x7f94200c8620 msgr2=0x7f942014e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:20.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 shutdown_connections 2024-08-18T13:51:20.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:20.095+0000 7f941ffff700 1 -- 172.21.15.118:0/1143507661 wait complete. 2024-08-18T13:51:20.101 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 41 2024-08-18T13:51:20.111 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-08-18T13:51:22.113 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:51:22.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8301255700 1 Processor -- start 2024-08-18T13:51:22.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8301255700 1 -- start start 2024-08-18T13:51:22.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8301255700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 0x7f82fc15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8301255700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc0c1110 con 0x7f82fc1556f0 2024-08-18T13:51:22.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8301255700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc0c1cb0 con 0x7f82fc0684e0 2024-08-18T13:51:22.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8301255700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c0b90 con 0x7f82fc1561d0 2024-08-18T13:51:22.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f82fc1556f0 0x7f82fc0c15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:54892/0 (socket says 172.21.15.118:54892) 2024-08-18T13:51:22.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 -- 172.21.15.118:0/3818508663 learned_addr learned my addr 172.21.15.118:0/3818508663 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:22.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8300a54700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 0x7f82fc15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597132809 0 0) 0x7f82fc0c1110 con 0x7f82fc1556f0 2024-08-18T13:51:22.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e4003570 con 0x7f82fc1556f0 2024-08-18T13:51:22.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530900010 0 0) 0x7f82fc0c1cb0 con 0x7f82fc0684e0 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82fc0c1110 con 0x7f82fc0684e0 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2852569321 0 0) 0x7f82e4003570 con 0x7f82fc1556f0 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 msgr2=0x7f82fc15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 0x7f82fc15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 >> v1:172.21.15.178:6789/0 conn(0x7f82fc0684e0 legacy=0x7f82fc155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0c8060 con 0x7f82fc1556f0 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7f82ec001470 con 0x7f82fc1556f0 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 >> v1:172.21.15.118:6789/0 conn(0x7f82fc1556f0 legacy=0x7f82fc0c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e4003c20 0x7f82e4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 0x7f82e40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c0b90 con 0x7f82e4003c20 2024-08-18T13:51:22.235 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e400bb90 con 0x7f82e4004810 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e400df60 con 0x7f82fc1561d0 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e4003c20 0x7f82e4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fb7fe700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f8300a54700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 0x7f82e40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 -- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 msgr2=0x7f82e40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc1561d0 0x7f82e40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 -- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82e4004810 msgr2=0x7f82e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 -- 172.21.15.118:0/3818508663 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4003570 con 0x7f82e4003c20 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fb7fe700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.227+0000 7f82fbfff700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e4003c20 0x7f82e4004030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c390 tx=0x7f82fc152f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6871c5dc694c388 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00cc60 con 0x7f82e4003c20 2024-08-18T13:51:22.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec00ce00 con 0x7f82e4003c20 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f82faffd700 1 -- 172.21.15.118:0/3818508663 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec005b70 con 0x7f82e4003c20 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- 172.21.15.118:0/3818508663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e4003c20 msgr2=0x7f82e4004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 --2- 172.21.15.118:0/3818508663 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82e4003c20 0x7f82e4004030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c390 tx=0x7f82fc152f60 comp rx=0 tx=0).stop 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 -- 172.21.15.118:0/3818508663 reap_dead start 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- 172.21.15.118:0/3818508663 shutdown_connections 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- 172.21.15.118:0/3818508663 >> 172.21.15.118:0/3818508663 conn(0x7f82fc06c020 msgr2=0x7f82fc0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- 172.21.15.118:0/3818508663 shutdown_connections 2024-08-18T13:51:22.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- 172.21.15.118:0/3818508663 wait complete. 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 Processor -- start 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- start start 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82fc06e310 0x7f82fc160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc06e6e0 0x7f82fc161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc143ea0 0x7f82fc1442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c76a0 con 0x7f82fc143ea0 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c2320 con 0x7f82fc06e310 2024-08-18T13:51:22.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8301255700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0c67b0 con 0x7f82fc06e6e0 2024-08-18T13:51:22.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc143ea0 0x7f82fc1442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc143ea0 0x7f82fc1442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:35584/0 (socket says 172.21.15.118:35584) 2024-08-18T13:51:22.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 -- 172.21.15.118:0/2885841332 learned_addr learned my addr 172.21.15.118:0/2885841332 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:22.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f82fb7fe700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc06e6e0 0x7f82fc161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f82fbfff700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82fc06e310 0x7f82fc160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.242 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 -- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc06e6e0 msgr2=0x7f82fc161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc06e6e0 0x7f82fc161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 -- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82fc06e310 msgr2=0x7f82fc160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82fc06e310 0x7f82fc160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f8300a54700 1 -- 172.21.15.118:0/2885841332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0c6a90 con 0x7f82fc143ea0 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.231+0000 7f82fbfff700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82fc06e310 0x7f82fc160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f8300a54700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc143ea0 0x7f82fc1442f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f82f40094d0 tx=0x7f82f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f82e3fff700 1 -- 172.21.15.118:0/2885841332 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f400e040 con 0x7f82fc143ea0 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f82e3fff700 1 -- 172.21.15.118:0/2885841332 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f400f040 con 0x7f82fc143ea0 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc0c6d70 con 0x7f82fc143ea0 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f82e3fff700 1 -- 172.21.15.118:0/2885841332 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f400fe40 con 0x7f82fc143ea0 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc0c7330 con 0x7f82fc143ea0 2024-08-18T13:51:22.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc0713a0 con 0x7f82fc143ea0 2024-08-18T13:51:22.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f82e3fff700 1 -- 172.21.15.118:0/2885841332 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f82f4015640 con 0x7f82fc143ea0 2024-08-18T13:51:22.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.235+0000 7f82e3fff700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82e8060860 0x7f82e8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:22.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.239+0000 7f82e3fff700 1 -- 172.21.15.118:0/2885841332 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f82f407dae0 con 0x7f82fc143ea0 2024-08-18T13:51:22.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.239+0000 7f82fbfff700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82e8060860 0x7f82e8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:22.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.239+0000 7f82fbfff700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82e8060860 0x7f82e8062d10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c390 tx=0x7f82ec012000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:22.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.243+0000 7f82e3fff700 1 -- 172.21.15.118:0/2885841332 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f82f404d970 con 0x7f82fc143ea0 2024-08-18T13:51:22.450 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.447+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82fc155c60 con 0x7f82fc143ea0 2024-08-18T13:51:22.451 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.447+0000 7f82e3fff700 1 -- 172.21.15.118:0/2885841332 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f82f407be60 con 0x7f82fc143ea0 2024-08-18T13:51:22.451 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:22.452 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5752,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2141281823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2141281823},{"type":"v1","addr":"172.21.15.178:6835","nonce":2141281823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:18.658279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":22,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82e8060860 msgr2=0x7f82e8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82e8060860 0x7f82e8062d10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c390 tx=0x7f82ec012000 comp rx=0 tx=0).stop 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc143ea0 msgr2=0x7f82fc1442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc143ea0 0x7f82fc1442f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f82f40094d0 tx=0x7f82f400b040 comp rx=0 tx=0).stop 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 shutdown_connections 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f82e8060860 0x7f82e8062d10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f82fc143ea0 0x7f82fc1442f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82fc06e6e0 0x7f82fc161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 --2- 172.21.15.118:0/2885841332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82fc06e310 0x7f82fc160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 >> 172.21.15.118:0/2885841332 conn(0x7f82fc06c020 msgr2=0x7f82fc146900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:22.454 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 shutdown_connections 2024-08-18T13:51:22.455 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:22.451+0000 7f8301255700 1 -- 172.21.15.118:0/2885841332 wait complete. 2024-08-18T13:51:22.455 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 42 2024-08-18T13:51:22.469 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-08-18T13:51:22.469 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 55 secs before thrashing 2024-08-18T13:51:27.784 INFO:tasks.ceph.mds.d.smithi178.stderr:2024-08-18T13:51:27.777+0000 7f6ef7530700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T13:51:44.407 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:51:44.410 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:51:49.746 INFO:tasks.workunit.client.0.smithi118.stderr:Updating files: 1% (256/12849) Updating files: 2% (257/12849) Updating files: 3% (386/12849) Updating files: 3% (446/12849) Updating files: 4% (514/12849) Updating files: 4% (640/12849) Updating files: 5% (643/12849) Updating files: 6% (771/12849) Updating files: 6% (866/12849) Updating files: 7% (900/12849) Updating files: 7% (1025/12849) Updating files: 8% (1028/12849) Updating files: 9% (1157/12849) Updating files: 9% (1186/12849) Updating files: 10% (1285/12849) Updating files: 10% (1367/12849) Updating files: 11% (1414/12849) Updating files: 12% (1542/12849) Updating files: 12% (1552/12849) Updating files: 13% (1671/12849) Updating files: 14% (1799/12849) Updating files: 14% (1807/12849) Updating files: 14% (1923/12849) Updating files: 15% (1928/12849) Updating files: 15% (2055/12849) Updating files: 16% (2056/12849) Updating files: 16% (2173/12849) Updating files: 17% (2185/12849) Updating files: 17% (2289/12849) Updating files: 18% (2313/12849) Updating files: 18% (2400/12849) Updating files: 19% (2442/12849) Updating files: 19% (2513/12849) Updating files: 20% (2570/12849) Updating files: 20% (2625/12849) Updating files: 21% (2699/12849) Updating files: 21% (2759/12849) Updating files: 22% (2827/12849) Updating files: 22% (2893/12849) Updating files: 23% (2956/12849) Updating files: 23% (3033/12849) Updating files: 24% (3084/12849) Updating files: 24% (3163/12849) Updating files: 25% (3213/12849) Updating files: 25% (3324/12849) Updating files: 26% (3341/12849) Updating files: 27% (3470/12849) Updating files: 27% (3484/12849) Updating files: 28% (3598/12849) Updating files: 28% (3626/12849) Updating files: 29% (3727/12849) Updating files: 29% (3771/12849) Updating files: 30% (3855/12849) Updating files: 30% (3909/12849) Updating files: 31% (3984/12849) Updating files: 31% (4043/12849) Updating files: 32% (4112/12849) Updating files: 32% (4192/12849) Updating files: 33% (4241/12849) Updating files: 33% (4335/12849) Updating files: 34% (4369/12849) Updating files: 34% (4455/12849) Updating files: 34% (4461/12849) Updating files: 34% (4462/12849) Updating files: 35% (4498/12849) Updating files: 35% (4601/12849) Updating files: 36% (4626/12849) Updating files: 37% (4755/12849) Updating files: 37% (4787/12849) Updating files: 38% (4883/12849) Updating files: 38% (5008/12849) Updating files: 39% (5012/12849) Updating files: 40% (5140/12849) Updating files: 40% (5196/12849) Updating files: 41% (5269/12849) Updating files: 41% (5385/12849) Updating files: 42% (5397/12849) Updating files: 42% (5507/12849) Updating files: 43% (5526/12849) Updating files: 44% (5654/12849) Updating files: 44% (5668/12849) Updating files: 45% (5783/12849) Updating files: 45% (5851/12849) Updating files: 46% (5911/12849) Updating files: 46% (6025/12849) Updating files: 47% (6040/12849) Updating files: 47% (6144/12849) Updating files: 48% (6168/12849) Updating files: 49% (6297/12849) Updating files: 49% (6350/12849) Updating files: 50% (6425/12849) Updating files: 50% (6531/12849) Updating files: 51% (6553/12849) Updating files: 52% (6682/12849) Updating files: 52% (6686/12849) Updating files: 53% (6810/12849) Updating files: 53% (6879/12849) Updating files: 54% (6939/12849) Updating files: 55% (7067/12849) Updating files: 55% (7088/12849) Updating files: 56% (7196/12849) Updating files: 56% (7294/12849) Updating files: 57% (7324/12849) Updating files: 58% (7453/12849) Updating files: 58% (7506/12849) Updating files: 59% (7581/12849) Updating files: 59% (7661/12849) Updating files: 60% (7710/12849) Updating files: 60% (7773/12849) Updating files: 61% (7838/12849) Updating files: 61% (7939/12849) Updating files: 62% (7967/12849) Updating files: 63% (8095/12849) Updating files: 63% (8138/12849) Updating files: 64% (8224/12849) Updating files: 64% (8295/12849) Updating files: 65% (8352/12849) Updating files: 65% (8448/12849) Updating files: 66% (8481/12849) Updating files: 66% (8570/12849) Updating files: 67% (8609/12849) Updating files: 67% (8674/12849) Updating files: 68% (8738/12849) Updating files: 68% (8769/12849) Updating files: 68% (8863/12849) Updating files: 69% (8866/12849) Updating files: 69% (8956/12849) Updating files: 70% (8995/12849) Updating files: 70% (9048/12849) Updating files: 71% (9123/12849) Updating files: 71% (9133/12849) Updating files: 71% (9236/12849) Updating files: 72% (9252/12849) Updating files: 72% (9290/12849) Updating files: 72% (9291/12849) Updating files: 73% (9380/12849) Updating files: 73% (9382/12849) Updating files: 74% (9509/12849) Updating files: 74% (9519/12849) Updating files: 74% (9629/12849) Updating files: 75% (9637/12849) Updating files: 75% (9745/12849) Updating files: 76% (9766/12849) Updating files: 76% (9880/12849) Updating files: 77% (9894/12849) Updating files: 78% (10023/12849) Updating files: 78% (10146/12849) Updating files: 79% (10151/12849) Updating files: 79% (10233/12849) Updating files: 79% (10251/12849) Updating files: 80% (10280/12849) Updating files: 80% (10329/12849) Updating files: 81% (10408/12849) Updating files: 81% (10472/12849) Updating files: 82% (10537/12849) Updating files: 82% (10637/12849) Updating files: 83% (10665/12849) Updating files: 84% (10794/12849) Updating files: 84% (10858/12849) Updating files: 84% (10890/12849) Updating files: 85% (10922/12849) Updating files: 86% (11051/12849) Updating files: 86% (11080/12849) Updating files: 87% (11179/12849) Updating files: 87% (11274/12849) Updating files: 88% (11308/12849) Updating files: 88% (11419/12849) Updating files: 89% (11436/12849) Updating files: 90% (11565/12849) Updating files: 90% (11569/12849) Updating files: 91% (11693/12849) Updating files: 91% (11760/12849) Updating files: 92% (11822/12849) Updating files: 93% (11950/12849) Updating files: 94% (12079/12849) Updating files: 94% (12135/12849) Updating files: 95% (12207/12849) Updating files: 95% (12319/12849) Updating files: 96% (12336/12849) Updating files: 97% (12464/12849) Updating files: 98% (12593/12849) Updating files: 98% (12666/12849) Updating files: 99% (12721/12849) Updating files: 100% (12849/12849) Updating files: 100% (12849/12849), done. 2024-08-18T13:51:49.901 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-08-18T13:51:49.901 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2024-08-18T13:51:49.902 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-18T13:51:50.157 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi118 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2024-08-18T13:51:50.888 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=011c2c82cd404891a9543c2fd0899f3c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a17c67c052c1476b20f99d847823755b26703128/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi118 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2024-08-18T13:51:50.891 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-08-18T13:51:50.902 INFO:tasks.mds_thrash:joining mds_thrasher 2024-08-18T13:51:50.904 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 0 2024-08-18T13:51:50.904 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2024-08-18T13:51:50.905 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 6 2024-08-18T13:51:50.905 INFO:tasks.mds_thrash:done joining 2024-08-18T13:51:50.905 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-08-18T13:51:50.925 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-08-18T13:51:50.926 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com... 2024-08-18T13:51:50.926 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-18T13:51:50.927 DEBUG:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-18T13:51:50.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi118.stderr:ceph-fuse[32128]: fuse finished with error 0 and tester_r 0 2024-08-18T13:51:51.009 INFO:teuthology.orchestra.run:waiting for 900 2024-08-18T13:51:57.111 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:51:57.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:51:57.111 DEBUG:teuthology.orchestra.run.smithi118:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-18T13:51:57.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:51:57.122 DEBUG:teuthology.orchestra.run.smithi118:> ip netns list 2024-08-18T13:51:57.129 INFO:teuthology.orchestra.run.smithi118.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-18T13:51:57.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:51:57.129 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-18T13:51:57.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T13:51:57.146 DEBUG:teuthology.orchestra.run.smithi118:> sudo ip link delete ceph-brx 2024-08-18T13:51:57.303 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-18T13:51:57.312 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-08-18T13:51:57.313 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:51:57.427 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfe31c700 1 Processor -- start 2024-08-18T13:51:57.428 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfe31c700 1 -- start start 2024-08-18T13:51:57.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfe31c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1cf80aef90 0x7f1cf80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfe31c700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf8155330 con 0x7f1cf80c6880 2024-08-18T13:51:57.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfe31c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf81555d0 con 0x7f1cf80aebc0 2024-08-18T13:51:57.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfe31c700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf8155140 con 0x7f1cf80aef90 2024-08-18T13:51:57.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfcb19700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1cf80aef90 0x7f1cf80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.429 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfcb19700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1cf80aef90 0x7f1cf80b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44880/0 (socket says 172.21.15.118:44880) 2024-08-18T13:51:57.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfcb19700 1 -- 172.21.15.118:0/2502264417 learned_addr learned my addr 172.21.15.118:0/2502264417 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:57.430 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.422+0000 7f1cfdb1b700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f1cf80c6880 0x7f1cf80c6c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38244/0 (socket says 172.21.15.118:38244) 2024-08-18T13:51:57.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfcb19700 1 -- 172.21.15.118:0/2502264417 >> v1:172.21.15.178:6789/0 conn(0x7f1cf80aebc0 legacy=0x7f1cf80abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfcb19700 1 -- 172.21.15.118:0/2502264417 >> v1:172.21.15.118:6789/0 conn(0x7f1cf80c6880 legacy=0x7f1cf80c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1ceffff700 1 -- 172.21.15.118:0/2502264417 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992083336 0 0) 0x7f1cf8155330 con 0x7f1cf80c6880 2024-08-18T13:51:57.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfcb19700 1 -- 172.21.15.118:0/2502264417 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8155870 con 0x7f1cf80aef90 2024-08-18T13:51:57.432 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfcb19700 1 --2- 172.21.15.118:0/2502264417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80aef90 0x7f1cf80b1640 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40099b0 tx=0x7f1cf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27173cec30e50b9d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:57.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1ceffff700 1 -- 172.21.15.118:0/2502264417 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400b860 con 0x7f1cf80aef90 2024-08-18T13:51:57.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1ceffff700 1 -- 172.21.15.118:0/2502264417 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400b9c0 con 0x7f1cf80aef90 2024-08-18T13:51:57.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1ceffff700 1 -- 172.21.15.118:0/2502264417 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400bd60 con 0x7f1cf80aef90 2024-08-18T13:51:57.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- 172.21.15.118:0/2502264417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80aef90 msgr2=0x7f1cf80b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.433 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/2502264417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80aef90 0x7f1cf80b1640 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40099b0 tx=0x7f1cf4009250 comp rx=0 tx=0).stop 2024-08-18T13:51:57.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- 172.21.15.118:0/2502264417 shutdown_connections 2024-08-18T13:51:57.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/2502264417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80aef90 0x7f1cf80b1640 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- 172.21.15.118:0/2502264417 >> 172.21.15.118:0/2502264417 conn(0x7f1cf80cbba0 msgr2=0x7f1cf80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:57.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- 172.21.15.118:0/2502264417 shutdown_connections 2024-08-18T13:51:57.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- 172.21.15.118:0/2502264417 wait complete. 2024-08-18T13:51:57.434 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 Processor -- start 2024-08-18T13:51:57.435 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- start start 2024-08-18T13:51:57.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1cf80aebc0 0x7f1cf81690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cf80aef90 0x7f1cf80ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.438 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80c6880 0x7f1cf80c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8169630 con 0x7f1cf80c6880 2024-08-18T13:51:57.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8152f70 con 0x7f1cf80aef90 2024-08-18T13:51:57.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfe31c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8165e30 con 0x7f1cf80aebc0 2024-08-18T13:51:57.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.426+0000 7f1cfdb1b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80c6880 0x7f1cf80c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80c6880 0x7f1cf80c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44896/0 (socket says 172.21.15.118:44896) 2024-08-18T13:51:57.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 -- 172.21.15.118:0/1102177379 learned_addr learned my addr 172.21.15.118:0/1102177379 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:57.439 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfcb19700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cf80aef90 0x7f1cf80ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.440 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfd31a700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1cf80aebc0 0x7f1cf81690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.440 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 -- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1cf80aebc0 msgr2=0x7f1cf81690f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.440 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1cf80aebc0 0x7f1cf81690f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.440 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 -- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cf80aef90 msgr2=0x7f1cf80ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.440 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cf80aef90 0x7f1cf80ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.440 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 -- 172.21.15.118:0/1102177379 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40094f0 con 0x7f1cf80c6880 2024-08-18T13:51:57.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfcb19700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cf80aef90 0x7f1cf80ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:51:57.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfdb1b700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80c6880 0x7f1cf80c41c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000cd00 tx=0x7f1cf0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:57.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1ced7fa700 1 -- 172.21.15.118:0/1102177379 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0005df0 con 0x7f1cf80c6880 2024-08-18T13:51:57.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1ced7fa700 1 -- 172.21.15.118:0/1102177379 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf000a960 con 0x7f1cf80c6880 2024-08-18T13:51:57.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf806a260 con 0x7f1cf80c6880 2024-08-18T13:51:57.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1ced7fa700 1 -- 172.21.15.118:0/1102177379 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000ad20 con 0x7f1cf80c6880 2024-08-18T13:51:57.441 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf81657e0 con 0x7f1cf80c6880 2024-08-18T13:51:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf80713a0 con 0x7f1cf80c6880 2024-08-18T13:51:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1ced7fa700 1 -- 172.21.15.118:0/1102177379 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1cf0007300 con 0x7f1cf80c6880 2024-08-18T13:51:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1ced7fa700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1cdc060860 0x7f1cdc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1ced7fa700 1 -- 172.21.15.118:0/1102177379 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f1cf007d310 con 0x7f1cf80c6880 2024-08-18T13:51:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.430+0000 7f1cfd31a700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1cdc060860 0x7f1cdc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.442 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.434+0000 7f1cfd31a700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1cdc060860 0x7f1cdc062d10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80037c0 tx=0x7f1ce8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:57.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.442+0000 7f1ced7fa700 1 -- 172.21.15.118:0/1102177379 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1cf004d220 con 0x7f1cf80c6880 2024-08-18T13:51:57.612 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.606+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cf80c9ab0 con 0x7f1cf80c6880 2024-08-18T13:51:57.614 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.610+0000 7f1ced7fa700 1 -- 172.21.15.118:0/1102177379 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18405 (secure 0 0 0) 0x7f1cf0050840 con 0x7f1cf80c6880 2024-08-18T13:51:57.615 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:57.615 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":54,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:50:50.639291+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/2054758182":"2024-08-19T13:50:50.639112+0000","172.21.15.178:6836/1054354354":"2024-08-19T13:49:30.622514+0000","172.21.15.178:6837/3452743394":"2024-08-19T13:45:35.570038+0000","172.21.15.178:6832/679522662":"2024-08-19T13:47:20.593054+0000","172.21.15.178:6836/3452743394":"2024-08-19T13:45:35.570038+0000","172.21.15.178:6832/1590569622":"2024-08-19T13:44:15.544758+0000","172.21.15.178:6832/2054758182":"2024-08-19T13:50:50.639112+0000","172.21.15.178:6837/1054354354":"2024-08-19T13:49:30.622514+0000","172.21.15.178:6835/679522662":"2024-08-19T13:47:20.593054+0000","172.21.15.178:6835/1590569622":"2024-08-19T13:44:15.544758+0000","172.21.15.178:6837/2503344632":"2024-08-19T13:42:15.510903+0000","172.21.15.178:6836/2503344632":"2024-08-19T13:42:15.510903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:51:57.617 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.610+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1cdc060860 msgr2=0x7f1cdc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.610+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1cdc060860 0x7f1cdc062d10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80037c0 tx=0x7f1ce8009040 comp rx=0 tx=0).stop 2024-08-18T13:51:57.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.610+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80c6880 msgr2=0x7f1cf80c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.610+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80c6880 0x7f1cf80c41c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000cd00 tx=0x7f1cf0001ac0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 shutdown_connections 2024-08-18T13:51:57.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1cdc060860 0x7f1cdc062d10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1cf80c6880 0x7f1cf80c41c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.618 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cf80aef90 0x7f1cf80ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 --2- 172.21.15.118:0/1102177379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1cf80aebc0 0x7f1cf81690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 >> 172.21.15.118:0/1102177379 conn(0x7f1cf80cbba0 msgr2=0x7f1cf80ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:57.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 shutdown_connections 2024-08-18T13:51:57.619 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.614+0000 7f1cfe31c700 1 -- 172.21.15.118:0/1102177379 wait complete. 2024-08-18T13:51:57.629 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-18T13:51:57.630 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:51:57.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 Processor -- start 2024-08-18T13:51:57.736 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 -- start start 2024-08-18T13:51:57.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80aebc0 0x7fcbf80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcbf80aef90 0x7fcbf80abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcbf80c6880 0x7fcbf80c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf8155140 con 0x7fcbf80aef90 2024-08-18T13:51:57.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf80af660 con 0x7fcbf80c6880 2024-08-18T13:51:57.738 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf8155330 con 0x7fcbf80aebc0 2024-08-18T13:51:57.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf5d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcbf80aef90 0x7fcbf80abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf5d9b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fcbf80aef90 0x7fcbf80abad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44900/0 (socket says 172.21.15.118:44900) 2024-08-18T13:51:57.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf5d9b700 1 -- 172.21.15.118:0/2437944243 learned_addr learned my addr 172.21.15.118:0/2437944243 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:57.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf6d9d700 1 --2- 172.21.15.118:0/2437944243 >> v2:172.21.15.178:3300/0 conn(0x7fcbf80c6880 0x7fcbf80c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.730+0000 7fcbf659c700 1 --2- 172.21.15.118:0/2437944243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80aebc0 0x7fcbf80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf5d9b700 1 -- 172.21.15.118:0/2437944243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80aebc0 msgr2=0x7fcbf80b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:57.739 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf5d9b700 1 --2- 172.21.15.118:0/2437944243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80aebc0 0x7fcbf80b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf5d9b700 1 -- 172.21.15.118:0/2437944243 >> v2:172.21.15.178:3300/0 conn(0x7fcbf80c6880 msgr2=0x7fcbf80c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:57.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf5d9b700 1 --2- 172.21.15.118:0/2437944243 >> v2:172.21.15.178:3300/0 conn(0x7fcbf80c6880 0x7fcbf80c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf5d9b700 1 -- 172.21.15.118:0/2437944243 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf81555d0 con 0x7fcbf80aef90 2024-08-18T13:51:57.740 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf5d9b700 1 --2- 172.21.15.118:0/2437944243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aef90 0x7fcbf80abad0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80ae730 tx=0x7fcbe8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21977b9a3a0a2e45 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:57.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf559a700 1 -- 172.21.15.118:0/2437944243 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800b860 con 0x7fcbf80aef90 2024-08-18T13:51:57.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf559a700 1 -- 172.21.15.118:0/2437944243 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe800b9c0 con 0x7fcbf80aef90 2024-08-18T13:51:57.743 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf559a700 1 -- 172.21.15.118:0/2437944243 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800bd60 con 0x7fcbf80aef90 2024-08-18T13:51:57.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 -- 172.21.15.118:0/2437944243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aef90 msgr2=0x7fcbf80abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 --2- 172.21.15.118:0/2437944243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aef90 0x7fcbf80abad0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80ae730 tx=0x7fcbe8009250 comp rx=0 tx=0).stop 2024-08-18T13:51:57.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 -- 172.21.15.118:0/2437944243 shutdown_connections 2024-08-18T13:51:57.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 --2- 172.21.15.118:0/2437944243 >> v2:172.21.15.178:3300/0 conn(0x7fcbf80c6880 0x7fcbf80c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 --2- 172.21.15.118:0/2437944243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aef90 0x7fcbf80abad0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.744 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 --2- 172.21.15.118:0/2437944243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80aebc0 0x7fcbf80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 -- 172.21.15.118:0/2437944243 >> 172.21.15.118:0/2437944243 conn(0x7fcbf80cbba0 msgr2=0x7fcbf80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:57.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 -- 172.21.15.118:0/2437944243 shutdown_connections 2024-08-18T13:51:57.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 -- 172.21.15.118:0/2437944243 wait complete. 2024-08-18T13:51:57.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 Processor -- start 2024-08-18T13:51:57.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.734+0000 7fcbf759e700 1 -- start start 2024-08-18T13:51:57.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aebc0 0x7fcbf8165010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.745 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbf80aef90 0x7fcbf8165550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80c6880 0x7fcbf80c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80c5920 con 0x7fcbf80aebc0 2024-08-18T13:51:57.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf8067480 con 0x7fcbf80aef90 2024-08-18T13:51:57.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80c3090 con 0x7fcbf80c6880 2024-08-18T13:51:57.746 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aebc0 0x7fcbf8165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aebc0 0x7fcbf8165010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44912/0 (socket says 172.21.15.118:44912) 2024-08-18T13:51:57.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 -- 172.21.15.118:0/3681497234 learned_addr learned my addr 172.21.15.118:0/3681497234 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:57.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf6d9d700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80c6880 0x7fcbf80c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf5d9b700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbf80aef90 0x7fcbf8165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 -- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80c6880 msgr2=0x7fcbf80c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80c6880 0x7fcbf80c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.747 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 -- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbf80aef90 msgr2=0x7fcbf8165550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbf80aef90 0x7fcbf8165550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 -- 172.21.15.118:0/3681497234 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80094f0 con 0x7fcbf80aebc0 2024-08-18T13:51:57.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf659c700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aebc0 0x7fcbf8165010 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00d890 tx=0x7fcbec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:57.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbe67fc700 1 -- 172.21.15.118:0/3681497234 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0125f0 con 0x7fcbf80aebc0 2024-08-18T13:51:57.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbe67fc700 1 -- 172.21.15.118:0/3681497234 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec012750 con 0x7fcbf80aebc0 2024-08-18T13:51:57.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf81583f0 con 0x7fcbf80aebc0 2024-08-18T13:51:57.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbe67fc700 1 -- 172.21.15.118:0/3681497234 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec012af0 con 0x7fcbf80aebc0 2024-08-18T13:51:57.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf80c5360 con 0x7fcbf80aebc0 2024-08-18T13:51:57.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.738+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf80c2e60 con 0x7fcbf80aebc0 2024-08-18T13:51:57.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.742+0000 7fcbe67fc700 1 -- 172.21.15.118:0/3681497234 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcbec01e460 con 0x7fcbf80aebc0 2024-08-18T13:51:57.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.742+0000 7fcbe67fc700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbd8060860 0x7fcbd8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:57.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.742+0000 7fcbe67fc700 1 -- 172.21.15.118:0/3681497234 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fcbec07d720 con 0x7fcbf80aebc0 2024-08-18T13:51:57.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.742+0000 7fcbf5d9b700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbd8060860 0x7fcbd8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:57.750 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.742+0000 7fcbf5d9b700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbd8060860 0x7fcbd8062d10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80ae730 tx=0x7fcbe8017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:57.756 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.750+0000 7fcbe67fc700 1 -- 172.21.15.118:0/3681497234 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcbf80c2e60 con 0x7fcbf80aebc0 2024-08-18T13:51:57.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.914+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbf80c31d0 con 0x7fcbd8060860 2024-08-18T13:51:57.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.934+0000 7fcbe67fc700 1 -- 172.21.15.118:0/3681497234 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+372617 (secure 0 0 0) 0x7fcbe800bb30 con 0x7fcbd8060860 2024-08-18T13:51:57.945 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:57.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.938+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbd8060860 msgr2=0x7fcbd8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.938+0000 7fcbf759e700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbd8060860 0x7fcbd8062d10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80ae730 tx=0x7fcbe8017000 comp rx=0 tx=0).stop 2024-08-18T13:51:57.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aebc0 msgr2=0x7fcbf8165010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:57.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aebc0 0x7fcbf8165010 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00d890 tx=0x7fcbec008040 comp rx=0 tx=0).stop 2024-08-18T13:51:57.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 shutdown_connections 2024-08-18T13:51:57.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbd8060860 0x7fcbd8062d10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.949 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbf80c6880 0x7fcbf80c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbf80aef90 0x7fcbf8165550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 --2- 172.21.15.118:0/3681497234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcbf80aebc0 0x7fcbf8165010 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:57.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 >> 172.21.15.118:0/3681497234 conn(0x7fcbf80cbba0 msgr2=0x7fcbf814eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:57.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 shutdown_connections 2024-08-18T13:51:57.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:57.942+0000 7fcbf759e700 1 -- 172.21.15.118:0/3681497234 wait complete. 2024-08-18T13:51:57.950 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:51:57.959 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-08-18T13:51:57.487883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1123528586,"num_objects":11047,"num_object_clones":0,"num_object_copies":43906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11047,"num_whiteouts":0,"num_read":22597,"num_read_kb":1390334,"num_write":28047,"num_write_kb":1250108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33741,"ondisk_log_size":33741,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4737384,"kb_used_data":2321128,"kb_used_omap":13,"kb_used_meta":2416178,"kb_avail":750237336,"statfs":{"total":773094113280,"available":768243032064,"internally_reserved":0,"allocated":2376835072,"data_stored":2374438700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2474166976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27132278,"num_objects":1405,"num_object_clones":0,"num_object_copies":5592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":9,"num_read_kb":916,"num_write":1574,"num_write_kb":27598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004102"},"pg_stats":[{"pgid":"2.1f","version":"53'151","reported_seq":214,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.743915+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:50:57.743915+0000","last_peered":"2024-08-18T13:50:57.743915+0000","last_clean":"2024-08-18T13:50:57.743915+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:50:57.743915+0000","last_undegraded":"2024-08-18T13:50:57.743915+0000","last_fullsized":"2024-08-18T13:50:57.743915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389178,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":28696,"num_write":150,"num_write_kb":8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766930+0000","last_change":"2024-08-18T13:43:55.555634+0000","last_active":"2024-08-18T13:51:51.766930+0000","last_peered":"2024-08-18T13:51:51.766930+0000","last_clean":"2024-08-18T13:51:51.766930+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:51:51.766930+0000","last_undegraded":"2024-08-18T13:51:51.766930+0000","last_fullsized":"2024-08-18T13:51:51.766930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064850389999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"54'676","reported_seq":3751,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.726737+0000","last_change":"2024-08-18T13:43:55.552107+0000","last_active":"2024-08-18T13:51:49.726737+0000","last_peered":"2024-08-18T13:51:49.726737+0000","last_clean":"2024-08-18T13:51:49.726737+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:51:49.726737+0000","last_undegraded":"2024-08-18T13:51:49.726737+0000","last_fullsized":"2024-08-18T13:51:49.726737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067303959999999996,"stat_sum":{"num_bytes":37566158,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":1013,"num_read_kb":36248,"num_write":673,"num_write_kb":41139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"54'78","reported_seq":126,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.670265+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:50:55.670265+0000","last_peered":"2024-08-18T13:50:55.670265+0000","last_clean":"2024-08-18T13:50:55.670265+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:50:55.670265+0000","last_undegraded":"2024-08-18T13:50:55.670265+0000","last_fullsized":"2024-08-18T13:50:55.670265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":75,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777966+0000","last_change":"2024-08-18T13:43:56.153208+0000","last_active":"2024-08-18T13:51:51.777966+0000","last_peered":"2024-08-18T13:51:51.777966+0000","last_clean":"2024-08-18T13:51:51.777966+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:51:51.777966+0000","last_undegraded":"2024-08-18T13:51:51.777966+0000","last_fullsized":"2024-08-18T13:51:51.777966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075430439999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"54'705","reported_seq":3559,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.547567+0000","last_change":"2024-08-18T13:43:55.555148+0000","last_active":"2024-08-18T13:51:49.547567+0000","last_peered":"2024-08-18T13:51:49.547567+0000","last_clean":"2024-08-18T13:51:49.547567+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:51:49.547567+0000","last_undegraded":"2024-08-18T13:51:49.547567+0000","last_fullsized":"2024-08-18T13:51:49.547567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075371290000000001,"stat_sum":{"num_bytes":29851183,"num_objects":370,"num_object_clones":0,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":792,"num_read_kb":20881,"num_write":708,"num_write_kb":37566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"54'194","reported_seq":259,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.297066+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:51:47.297066+0000","last_peered":"2024-08-18T13:51:47.297066+0000","last_clean":"2024-08-18T13:51:47.297066+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:51:47.297066+0000","last_undegraded":"2024-08-18T13:51:47.297066+0000","last_fullsized":"2024-08-18T13:51:47.297066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":69650,"num_write":192,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775007+0000","last_change":"2024-08-18T13:43:55.857350+0000","last_active":"2024-08-18T13:51:51.775007+0000","last_peered":"2024-08-18T13:51:51.775007+0000","last_clean":"2024-08-18T13:51:51.775007+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:51:51.775007+0000","last_undegraded":"2024-08-18T13:51:51.775007+0000","last_fullsized":"2024-08-18T13:51:51.775007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046085359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"54'680","reported_seq":3480,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.425160+0000","last_change":"2024-08-18T13:43:55.557895+0000","last_active":"2024-08-18T13:51:49.425160+0000","last_peered":"2024-08-18T13:51:49.425160+0000","last_clean":"2024-08-18T13:51:49.425160+0000","last_became_active":"2024-08-18T13:40:16.608743+0000","last_became_peered":"2024-08-18T13:40:16.608743+0000","last_unstale":"2024-08-18T13:51:49.425160+0000","last_undegraded":"2024-08-18T13:51:49.425160+0000","last_fullsized":"2024-08-18T13:51:49.425160+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:59:49.527489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080275980000000004,"stat_sum":{"num_bytes":28774140,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":727,"num_read_kb":13856,"num_write":680,"num_write_kb":28426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"53'23","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.459383+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:50:56.459383+0000","last_peered":"2024-08-18T13:50:56.459383+0000","last_clean":"2024-08-18T13:50:56.459383+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:50:56.459383+0000","last_undegraded":"2024-08-18T13:50:56.459383+0000","last_fullsized":"2024-08-18T13:50:56.459383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":25,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777014+0000","last_change":"2024-08-18T13:43:55.862914+0000","last_active":"2024-08-18T13:51:51.777014+0000","last_peered":"2024-08-18T13:51:51.777014+0000","last_clean":"2024-08-18T13:51:51.777014+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:51:51.777014+0000","last_undegraded":"2024-08-18T13:51:51.777014+0000","last_fullsized":"2024-08-18T13:51:51.777014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065299970000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"54'674","reported_seq":2902,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.448739+0000","last_change":"2024-08-18T13:43:55.555499+0000","last_active":"2024-08-18T13:51:49.448739+0000","last_peered":"2024-08-18T13:51:49.448739+0000","last_clean":"2024-08-18T13:51:49.448739+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:51:49.448739+0000","last_undegraded":"2024-08-18T13:51:49.448739+0000","last_fullsized":"2024-08-18T13:51:49.448739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078514629999999995,"stat_sum":{"num_bytes":11851705,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":166,"num_read_kb":1355,"num_write":673,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755374+0000","last_change":"2024-08-18T13:43:55.954106+0000","last_active":"2024-08-18T13:51:51.755374+0000","last_peered":"2024-08-18T13:51:51.755374+0000","last_clean":"2024-08-18T13:51:51.755374+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:51:51.755374+0000","last_undegraded":"2024-08-18T13:51:51.755374+0000","last_fullsized":"2024-08-18T13:51:51.755374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033699279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"54'215","reported_seq":274,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:53.770156+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:51:53.770156+0000","last_peered":"2024-08-18T13:51:53.770156+0000","last_clean":"2024-08-18T13:51:53.770156+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:51:53.770156+0000","last_undegraded":"2024-08-18T13:51:53.770156+0000","last_fullsized":"2024-08-18T13:51:53.770156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11951501,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":23,"num_read_kb":45068,"num_write":210,"num_write_kb":11823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"54'638","reported_seq":3281,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.214771+0000","last_change":"2024-08-18T13:43:55.558893+0000","last_active":"2024-08-18T13:51:49.214771+0000","last_peered":"2024-08-18T13:51:49.214771+0000","last_clean":"2024-08-18T13:51:49.214771+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:51:49.214771+0000","last_undegraded":"2024-08-18T13:51:49.214771+0000","last_fullsized":"2024-08-18T13:51:49.214771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059019540000000001,"stat_sum":{"num_bytes":16782769,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":581,"num_read_kb":14450,"num_write":639,"num_write_kb":24822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763887+0000","last_change":"2024-08-18T13:43:55.554204+0000","last_active":"2024-08-18T13:51:51.763887+0000","last_peered":"2024-08-18T13:51:51.763887+0000","last_clean":"2024-08-18T13:51:51.763887+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:51:51.763887+0000","last_undegraded":"2024-08-18T13:51:51.763887+0000","last_fullsized":"2024-08-18T13:51:51.763887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074252320000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"54'28","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.687997+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:50:56.687997+0000","last_peered":"2024-08-18T13:50:56.687997+0000","last_clean":"2024-08-18T13:50:56.687997+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:50:56.687997+0000","last_undegraded":"2024-08-18T13:50:56.687997+0000","last_fullsized":"2024-08-18T13:50:56.687997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":28,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"54'709","reported_seq":3799,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.722324+0000","last_change":"2024-08-18T13:43:55.558557+0000","last_active":"2024-08-18T13:51:49.722324+0000","last_peered":"2024-08-18T13:51:49.722324+0000","last_clean":"2024-08-18T13:51:49.722324+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:51:49.722324+0000","last_undegraded":"2024-08-18T13:51:49.722324+0000","last_fullsized":"2024-08-18T13:51:49.722324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057029610000000003,"stat_sum":{"num_bytes":33413248,"num_objects":347,"num_object_clones":0,"num_object_copies":1388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1028,"num_read_kb":27410,"num_write":707,"num_write_kb":41185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761963+0000","last_change":"2024-08-18T13:43:55.553624+0000","last_active":"2024-08-18T13:51:51.761963+0000","last_peered":"2024-08-18T13:51:51.761963+0000","last_clean":"2024-08-18T13:51:51.761963+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:51:51.761963+0000","last_undegraded":"2024-08-18T13:51:51.761963+0000","last_fullsized":"2024-08-18T13:51:51.761963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069243719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"53'13","reported_seq":57,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:50.648992+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:50:50.648992+0000","last_peered":"2024-08-18T13:50:50.648992+0000","last_clean":"2024-08-18T13:50:50.648992+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:50:50.648992+0000","last_undegraded":"2024-08-18T13:50:50.648992+0000","last_fullsized":"2024-08-18T13:50:50.648992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":11,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'683","reported_seq":3570,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.650342+0000","last_change":"2024-08-18T13:43:56.142589+0000","last_active":"2024-08-18T13:51:49.650342+0000","last_peered":"2024-08-18T13:51:49.650342+0000","last_clean":"2024-08-18T13:51:49.650342+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:51:49.650342+0000","last_undegraded":"2024-08-18T13:51:49.650342+0000","last_fullsized":"2024-08-18T13:51:49.650342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079287170000000001,"stat_sum":{"num_bytes":20263748,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":825,"num_read_kb":25932,"num_write":679,"num_write_kb":28331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:50.650367+0000","last_change":"2024-08-18T13:43:55.558187+0000","last_active":"2024-08-18T13:50:50.650367+0000","last_peered":"2024-08-18T13:50:50.650367+0000","last_clean":"2024-08-18T13:50:50.650367+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:50:50.650367+0000","last_undegraded":"2024-08-18T13:50:50.650367+0000","last_fullsized":"2024-08-18T13:50:50.650367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068791520000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"54'106","reported_seq":153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.256673+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:51:41.256673+0000","last_peered":"2024-08-18T13:51:41.256673+0000","last_clean":"2024-08-18T13:51:41.256673+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:51:41.256673+0000","last_undegraded":"2024-08-18T13:51:41.256673+0000","last_fullsized":"2024-08-18T13:51:41.256673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":108,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"54'617","reported_seq":3452,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.520510+0000","last_change":"2024-08-18T13:43:55.559559+0000","last_active":"2024-08-18T13:51:49.520510+0000","last_peered":"2024-08-18T13:51:49.520510+0000","last_clean":"2024-08-18T13:51:49.520510+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:51:49.520510+0000","last_undegraded":"2024-08-18T13:51:49.520510+0000","last_fullsized":"2024-08-18T13:51:49.520510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004078028,"stat_sum":{"num_bytes":24272349,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":773,"num_read_kb":25488,"num_write":617,"num_write_kb":28093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'93","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762461+0000","last_change":"2024-08-18T13:43:56.154543+0000","last_active":"2024-08-18T13:51:51.762461+0000","last_peered":"2024-08-18T13:51:51.762461+0000","last_clean":"2024-08-18T13:51:51.762461+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:51:51.762461+0000","last_undegraded":"2024-08-18T13:51:51.762461+0000","last_fullsized":"2024-08-18T13:51:51.762461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060661889999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"54'111","reported_seq":160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.305665+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:51:17.305665+0000","last_peered":"2024-08-18T13:51:17.305665+0000","last_clean":"2024-08-18T13:51:17.305665+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:51:17.305665+0000","last_undegraded":"2024-08-18T13:51:17.305665+0000","last_fullsized":"2024-08-18T13:51:17.305665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":111,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"54'720","reported_seq":2969,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.297825+0000","last_change":"2024-08-18T13:43:55.557702+0000","last_active":"2024-08-18T13:51:49.297825+0000","last_peered":"2024-08-18T13:51:49.297825+0000","last_clean":"2024-08-18T13:51:49.297825+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:51:49.297825+0000","last_undegraded":"2024-08-18T13:51:49.297825+0000","last_fullsized":"2024-08-18T13:51:49.297825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079796720000000002,"stat_sum":{"num_bytes":17871290,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":187,"num_read_kb":7768,"num_write":717,"num_write_kb":21887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781486+0000","last_change":"2024-08-18T13:43:55.555491+0000","last_active":"2024-08-18T13:51:51.781486+0000","last_peered":"2024-08-18T13:51:51.781486+0000","last_clean":"2024-08-18T13:51:51.781486+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:51:51.781486+0000","last_undegraded":"2024-08-18T13:51:51.781486+0000","last_fullsized":"2024-08-18T13:51:51.781486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054340660000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"54'66","reported_seq":116,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:48.746876+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:51:48.746876+0000","last_peered":"2024-08-18T13:51:48.746876+0000","last_clean":"2024-08-18T13:51:48.746876+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:51:48.746876+0000","last_undegraded":"2024-08-18T13:51:48.746876+0000","last_fullsized":"2024-08-18T13:51:48.746876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":184,"num_write":164,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"54'714","reported_seq":2929,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.710846+0000","last_change":"2024-08-18T13:43:55.550748+0000","last_active":"2024-08-18T13:51:49.710846+0000","last_peered":"2024-08-18T13:51:49.710846+0000","last_clean":"2024-08-18T13:51:49.710846+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:51:49.710846+0000","last_undegraded":"2024-08-18T13:51:49.710846+0000","last_fullsized":"2024-08-18T13:51:49.710846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057306029999999999,"stat_sum":{"num_bytes":13242151,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":153,"num_read_kb":5341,"num_write":712,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758218+0000","last_change":"2024-08-18T13:43:56.143836+0000","last_active":"2024-08-18T13:51:51.758218+0000","last_peered":"2024-08-18T13:51:51.758218+0000","last_clean":"2024-08-18T13:51:51.758218+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:51:51.758218+0000","last_undegraded":"2024-08-18T13:51:51.758218+0000","last_fullsized":"2024-08-18T13:51:51.758218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064599770000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"54'289","reported_seq":337,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:33.863944+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:51:33.863944+0000","last_peered":"2024-08-18T13:51:33.863944+0000","last_clean":"2024-08-18T13:51:33.863944+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:51:33.863944+0000","last_undegraded":"2024-08-18T13:51:33.863944+0000","last_fullsized":"2024-08-18T13:51:33.863944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":12176,"num_write":284,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"54'704","reported_seq":3425,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.623916+0000","last_change":"2024-08-18T13:43:56.151855+0000","last_active":"2024-08-18T13:51:49.623916+0000","last_peered":"2024-08-18T13:51:49.623916+0000","last_clean":"2024-08-18T13:51:49.623916+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:51:49.623916+0000","last_undegraded":"2024-08-18T13:51:49.623916+0000","last_fullsized":"2024-08-18T13:51:49.623916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060004689999999996,"stat_sum":{"num_bytes":28446730,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":659,"num_read_kb":21792,"num_write":702,"num_write_kb":28155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"54'84","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775443+0000","last_change":"2024-08-18T13:43:55.863531+0000","last_active":"2024-08-18T13:51:51.775443+0000","last_peered":"2024-08-18T13:51:51.775443+0000","last_clean":"2024-08-18T13:51:51.775443+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:51:51.775443+0000","last_undegraded":"2024-08-18T13:51:51.775443+0000","last_fullsized":"2024-08-18T13:51:51.775443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006455484,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"54'657","reported_seq":700,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.994466+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:51:38.994466+0000","last_peered":"2024-08-18T13:51:38.994466+0000","last_clean":"2024-08-18T13:51:38.994466+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:51:38.994466+0000","last_undegraded":"2024-08-18T13:51:38.994466+0000","last_fullsized":"2024-08-18T13:51:38.994466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"54'691","reported_seq":3343,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.557816+0000","last_change":"2024-08-18T13:43:56.141004+0000","last_active":"2024-08-18T13:51:49.557816+0000","last_peered":"2024-08-18T13:51:49.557816+0000","last_clean":"2024-08-18T13:51:49.557816+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:51:49.557816+0000","last_undegraded":"2024-08-18T13:51:49.557816+0000","last_fullsized":"2024-08-18T13:51:49.557816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067583870000000002,"stat_sum":{"num_bytes":38683931,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":590,"num_read_kb":16518,"num_write":694,"num_write_kb":42167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"54'97","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758746+0000","last_change":"2024-08-18T13:43:55.559256+0000","last_active":"2024-08-18T13:51:51.758746+0000","last_peered":"2024-08-18T13:51:51.758746+0000","last_clean":"2024-08-18T13:51:51.758746+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:51:51.758746+0000","last_undegraded":"2024-08-18T13:51:51.758746+0000","last_fullsized":"2024-08-18T13:51:51.758746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005854756,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"53'383","reported_seq":438,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.690458+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:50:55.690458+0000","last_peered":"2024-08-18T13:50:55.690458+0000","last_clean":"2024-08-18T13:50:55.690458+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:50:55.690458+0000","last_undegraded":"2024-08-18T13:50:55.690458+0000","last_fullsized":"2024-08-18T13:50:55.690458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":43191,"num_write":382,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"54'661","reported_seq":4460,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.576473+0000","last_change":"2024-08-18T13:43:56.144223+0000","last_active":"2024-08-18T13:51:49.576473+0000","last_peered":"2024-08-18T13:51:49.576473+0000","last_clean":"2024-08-18T13:51:49.576473+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:51:49.576473+0000","last_undegraded":"2024-08-18T13:51:49.576473+0000","last_fullsized":"2024-08-18T13:51:49.576473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067969390000000001,"stat_sum":{"num_bytes":51061167,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":1737,"num_read_kb":43040,"num_write":661,"num_write_kb":53481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"54'83","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758102+0000","last_change":"2024-08-18T13:43:56.143663+0000","last_active":"2024-08-18T13:51:51.758102+0000","last_peered":"2024-08-18T13:51:51.758102+0000","last_clean":"2024-08-18T13:51:51.758102+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:51:51.758102+0000","last_undegraded":"2024-08-18T13:51:51.758102+0000","last_fullsized":"2024-08-18T13:51:51.758102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063915389999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"53'120","reported_seq":283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.353455+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:50:51.353455+0000","last_peered":"2024-08-18T13:50:51.353455+0000","last_clean":"2024-08-18T13:50:51.353455+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:50:51.353455+0000","last_undegraded":"2024-08-18T13:50:51.353455+0000","last_fullsized":"2024-08-18T13:50:51.353455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":638,"num_write":117,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"54'643","reported_seq":3597,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.312624+0000","last_change":"2024-08-18T13:43:56.143097+0000","last_active":"2024-08-18T13:51:49.312624+0000","last_peered":"2024-08-18T13:51:49.312624+0000","last_clean":"2024-08-18T13:51:49.312624+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:51:49.312624+0000","last_undegraded":"2024-08-18T13:51:49.312624+0000","last_fullsized":"2024-08-18T13:51:49.312624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079242040000000007,"stat_sum":{"num_bytes":41790406,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":892,"num_read_kb":23319,"num_write":639,"num_write_kb":45248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'82","reported_seq":2146,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762481+0000","last_change":"2024-08-18T13:43:56.149372+0000","last_active":"2024-08-18T13:51:51.762481+0000","last_peered":"2024-08-18T13:51:51.762481+0000","last_clean":"2024-08-18T13:51:51.762481+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:51:51.762481+0000","last_undegraded":"2024-08-18T13:51:51.762481+0000","last_fullsized":"2024-08-18T13:51:51.762481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041495459999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"54'234","reported_seq":290,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.626190+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:50:57.626190+0000","last_peered":"2024-08-18T13:50:57.626190+0000","last_clean":"2024-08-18T13:50:57.626190+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:50:57.626190+0000","last_undegraded":"2024-08-18T13:50:57.626190+0000","last_fullsized":"2024-08-18T13:50:57.626190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":49152,"num_write":231,"num_write_kb":12421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"54'716","reported_seq":3216,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.491284+0000","last_change":"2024-08-18T13:43:56.153929+0000","last_active":"2024-08-18T13:51:49.491284+0000","last_peered":"2024-08-18T13:51:49.491284+0000","last_clean":"2024-08-18T13:51:49.491284+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:51:49.491284+0000","last_undegraded":"2024-08-18T13:51:49.491284+0000","last_fullsized":"2024-08-18T13:51:49.491284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058974379999999996,"stat_sum":{"num_bytes":30462084,"num_objects":352,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":439,"num_read_kb":15189,"num_write":710,"num_write_kb":30067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779966+0000","last_change":"2024-08-18T13:43:55.552534+0000","last_active":"2024-08-18T13:51:51.779966+0000","last_peered":"2024-08-18T13:51:51.779966+0000","last_clean":"2024-08-18T13:51:51.779966+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:51:51.779966+0000","last_undegraded":"2024-08-18T13:51:51.779966+0000","last_fullsized":"2024-08-18T13:51:51.779966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068182370000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"54'982","reported_seq":1027,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.365615+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:51:17.365615+0000","last_peered":"2024-08-18T13:51:17.365615+0000","last_clean":"2024-08-18T13:51:17.365615+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:51:17.365615+0000","last_undegraded":"2024-08-18T13:51:17.365615+0000","last_fullsized":"2024-08-18T13:51:17.365615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2087,"num_write":979,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:49:30.943967+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:49:30.943967+0000","last_peered":"2024-08-18T13:49:30.943967+0000","last_clean":"2024-08-18T13:49:30.943967+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:49:30.943967+0000","last_undegraded":"2024-08-18T13:49:30.943967+0000","last_fullsized":"2024-08-18T13:49:30.943967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"54'664","reported_seq":3524,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.497597+0000","last_change":"2024-08-18T13:43:55.956673+0000","last_active":"2024-08-18T13:51:49.497597+0000","last_peered":"2024-08-18T13:51:49.497597+0000","last_clean":"2024-08-18T13:51:49.497597+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:51:49.497597+0000","last_undegraded":"2024-08-18T13:51:49.497597+0000","last_fullsized":"2024-08-18T13:51:49.497597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035684480000000001,"stat_sum":{"num_bytes":39930170,"num_objects":364,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":798,"num_read_kb":26403,"num_write":662,"num_write_kb":47533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779241+0000","last_change":"2024-08-18T13:43:55.552313+0000","last_active":"2024-08-18T13:51:51.779241+0000","last_peered":"2024-08-18T13:51:51.779241+0000","last_clean":"2024-08-18T13:51:51.779241+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:51:51.779241+0000","last_undegraded":"2024-08-18T13:51:51.779241+0000","last_fullsized":"2024-08-18T13:51:51.779241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057711639999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"54'76","reported_seq":118,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.291821+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:51:47.291821+0000","last_peered":"2024-08-18T13:51:47.291821+0000","last_clean":"2024-08-18T13:51:47.291821+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:51:47.291821+0000","last_undegraded":"2024-08-18T13:51:47.291821+0000","last_fullsized":"2024-08-18T13:51:47.291821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"54'688","reported_seq":4127,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.578882+0000","last_change":"2024-08-18T13:43:55.953251+0000","last_active":"2024-08-18T13:51:49.578882+0000","last_peered":"2024-08-18T13:51:49.578882+0000","last_clean":"2024-08-18T13:51:49.578882+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:51:49.578882+0000","last_undegraded":"2024-08-18T13:51:49.578882+0000","last_fullsized":"2024-08-18T13:51:49.578882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032433420000000002,"stat_sum":{"num_bytes":54557211,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1377,"num_read_kb":50055,"num_write":687,"num_write_kb":57734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"54'95","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.756325+0000","last_change":"2024-08-18T13:43:55.918416+0000","last_active":"2024-08-18T13:51:51.756325+0000","last_peered":"2024-08-18T13:51:51.756325+0000","last_clean":"2024-08-18T13:51:51.756325+0000","last_became_active":"2024-08-18T13:40:16.614814+0000","last_became_peered":"2024-08-18T13:40:16.614814+0000","last_unstale":"2024-08-18T13:51:51.756325+0000","last_undegraded":"2024-08-18T13:51:51.756325+0000","last_fullsized":"2024-08-18T13:51:51.756325+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:02:39.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033975450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'31","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.186584+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:50:56.186584+0000","last_peered":"2024-08-18T13:50:56.186584+0000","last_clean":"2024-08-18T13:50:56.186584+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:50:56.186584+0000","last_undegraded":"2024-08-18T13:50:56.186584+0000","last_fullsized":"2024-08-18T13:50:56.186584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":33,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"54'670","reported_seq":3234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.448131+0000","last_change":"2024-08-18T13:43:56.153281+0000","last_active":"2024-08-18T13:51:49.448131+0000","last_peered":"2024-08-18T13:51:49.448131+0000","last_clean":"2024-08-18T13:51:49.448131+0000","last_became_active":"2024-08-18T13:40:16.609908+0000","last_became_peered":"2024-08-18T13:40:16.609908+0000","last_unstale":"2024-08-18T13:51:49.448131+0000","last_undegraded":"2024-08-18T13:51:49.448131+0000","last_fullsized":"2024-08-18T13:51:49.448131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:35:52.383892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069871079999999997,"stat_sum":{"num_bytes":26719276,"num_objects":358,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":491,"num_read_kb":18148,"num_write":671,"num_write_kb":26414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4,5],"acting":[0,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784591+0000","last_change":"2024-08-18T13:43:55.553120+0000","last_active":"2024-08-18T13:51:51.784591+0000","last_peered":"2024-08-18T13:51:51.784591+0000","last_clean":"2024-08-18T13:51:51.784591+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:51:51.784591+0000","last_undegraded":"2024-08-18T13:51:51.784591+0000","last_fullsized":"2024-08-18T13:51:51.784591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057951929999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"53'29","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.394499+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:50:56.394499+0000","last_peered":"2024-08-18T13:50:56.394499+0000","last_clean":"2024-08-18T13:50:56.394499+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:50:56.394499+0000","last_undegraded":"2024-08-18T13:50:56.394499+0000","last_fullsized":"2024-08-18T13:50:56.394499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":30,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"54'688","reported_seq":2818,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.687818+0000","last_change":"2024-08-18T13:43:55.859666+0000","last_active":"2024-08-18T13:51:49.687818+0000","last_peered":"2024-08-18T13:51:49.687818+0000","last_clean":"2024-08-18T13:51:49.687818+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:51:49.687818+0000","last_undegraded":"2024-08-18T13:51:49.687818+0000","last_fullsized":"2024-08-18T13:51:49.687818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070733389999999997,"stat_sum":{"num_bytes":17282460,"num_objects":346,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":68,"num_read_kb":8224,"num_write":686,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"54'66","reported_seq":2129,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783626+0000","last_change":"2024-08-18T13:43:55.551809+0000","last_active":"2024-08-18T13:51:51.783626+0000","last_peered":"2024-08-18T13:51:51.783626+0000","last_clean":"2024-08-18T13:51:51.783626+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:51:51.783626+0000","last_undegraded":"2024-08-18T13:51:51.783626+0000","last_fullsized":"2024-08-18T13:51:51.783626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060906930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"53'26","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.647720+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:50:55.647720+0000","last_peered":"2024-08-18T13:50:55.647720+0000","last_clean":"2024-08-18T13:50:55.647720+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:50:55.647720+0000","last_undegraded":"2024-08-18T13:50:55.647720+0000","last_fullsized":"2024-08-18T13:50:55.647720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":27,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"54'695","reported_seq":3630,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.666776+0000","last_change":"2024-08-18T13:43:55.557389+0000","last_active":"2024-08-18T13:51:49.666776+0000","last_peered":"2024-08-18T13:51:49.666776+0000","last_clean":"2024-08-18T13:51:49.666776+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:51:49.666776+0000","last_undegraded":"2024-08-18T13:51:49.666776+0000","last_fullsized":"2024-08-18T13:51:49.666776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011606655,"stat_sum":{"num_bytes":44119789,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":873,"num_read_kb":37157,"num_write":696,"num_write_kb":47501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"54'91","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784041+0000","last_change":"2024-08-18T13:43:55.553915+0000","last_active":"2024-08-18T13:51:51.784041+0000","last_peered":"2024-08-18T13:51:51.784041+0000","last_clean":"2024-08-18T13:51:51.784041+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:51:51.784041+0000","last_undegraded":"2024-08-18T13:51:51.784041+0000","last_fullsized":"2024-08-18T13:51:51.784041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053019490000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"53'1204","reported_seq":1276,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.252643+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:51:17.252643+0000","last_peered":"2024-08-18T13:51:17.252643+0000","last_clean":"2024-08-18T13:51:17.252643+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:51:17.252643+0000","last_undegraded":"2024-08-18T13:51:17.252643+0000","last_fullsized":"2024-08-18T13:51:17.252643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":78,"num_read_kb":12399,"num_write":1202,"num_write_kb":8830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'657","reported_seq":3516,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.665369+0000","last_change":"2024-08-18T13:43:56.154837+0000","last_active":"2024-08-18T13:51:49.665369+0000","last_peered":"2024-08-18T13:51:49.665369+0000","last_clean":"2024-08-18T13:51:49.665369+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:51:49.665369+0000","last_undegraded":"2024-08-18T13:51:49.665369+0000","last_fullsized":"2024-08-18T13:51:49.665369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054104280000000001,"stat_sum":{"num_bytes":33280018,"num_objects":296,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296,"num_whiteouts":0,"num_read":797,"num_read_kb":22024,"num_write":655,"num_write_kb":32783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757105+0000","last_change":"2024-08-18T13:43:55.959459+0000","last_active":"2024-08-18T13:51:51.757105+0000","last_peered":"2024-08-18T13:51:51.757105+0000","last_clean":"2024-08-18T13:51:51.757105+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:51:51.757105+0000","last_undegraded":"2024-08-18T13:51:51.757105+0000","last_fullsized":"2024-08-18T13:51:51.757105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044255090000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"54'5","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.846481+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:50:51.846481+0000","last_peered":"2024-08-18T13:50:51.846481+0000","last_clean":"2024-08-18T13:50:51.846481+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:50:51.846481+0000","last_undegraded":"2024-08-18T13:50:51.846481+0000","last_fullsized":"2024-08-18T13:50:51.846481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"54'660","reported_seq":2779,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.587481+0000","last_change":"2024-08-18T13:43:55.555242+0000","last_active":"2024-08-18T13:51:49.587481+0000","last_peered":"2024-08-18T13:51:49.587481+0000","last_clean":"2024-08-18T13:51:49.587481+0000","last_became_active":"2024-08-18T13:40:16.915369+0000","last_became_peered":"2024-08-18T13:40:16.915369+0000","last_unstale":"2024-08-18T13:51:49.587481+0000","last_undegraded":"2024-08-18T13:51:49.587481+0000","last_fullsized":"2024-08-18T13:51:49.587481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:50:23.633304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005619059,"stat_sum":{"num_bytes":16977484,"num_objects":353,"num_object_clones":0,"num_object_copies":1412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":46,"num_read_kb":4277,"num_write":660,"num_write_kb":20975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,6],"acting":[3,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"54'101","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784667+0000","last_change":"2024-08-18T13:43:55.551514+0000","last_active":"2024-08-18T13:51:51.784667+0000","last_peered":"2024-08-18T13:51:51.784667+0000","last_clean":"2024-08-18T13:51:51.784667+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:51:51.784667+0000","last_undegraded":"2024-08-18T13:51:51.784667+0000","last_fullsized":"2024-08-18T13:51:51.784667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064019180000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"53'2","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:50:28.547615+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:50:28.547615+0000","last_peered":"2024-08-18T13:50:28.547615+0000","last_clean":"2024-08-18T13:50:28.547615+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:50:28.547615+0000","last_undegraded":"2024-08-18T13:50:28.547615+0000","last_fullsized":"2024-08-18T13:50:28.547615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"54'625","reported_seq":3598,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.686816+0000","last_change":"2024-08-18T13:43:55.554811+0000","last_active":"2024-08-18T13:51:49.686816+0000","last_peered":"2024-08-18T13:51:49.686816+0000","last_clean":"2024-08-18T13:51:49.686816+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:51:49.686816+0000","last_undegraded":"2024-08-18T13:51:49.686816+0000","last_fullsized":"2024-08-18T13:51:49.686816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074125900000000002,"stat_sum":{"num_bytes":40693231,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":911,"num_read_kb":37155,"num_write":622,"num_write_kb":48255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778223+0000","last_change":"2024-08-18T13:43:56.154386+0000","last_active":"2024-08-18T13:51:51.778223+0000","last_peered":"2024-08-18T13:51:51.778223+0000","last_clean":"2024-08-18T13:51:51.778223+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:51:51.778223+0000","last_undegraded":"2024-08-18T13:51:51.778223+0000","last_fullsized":"2024-08-18T13:51:51.778223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059765870000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"54'216","reported_seq":283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.281097+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:51:41.281097+0000","last_peered":"2024-08-18T13:51:41.281097+0000","last_clean":"2024-08-18T13:51:41.281097+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:51:41.281097+0000","last_undegraded":"2024-08-18T13:51:41.281097+0000","last_fullsized":"2024-08-18T13:51:41.281097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":45098,"num_write":215,"num_write_kb":8331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'629","reported_seq":3941,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.453295+0000","last_change":"2024-08-18T13:43:55.862627+0000","last_active":"2024-08-18T13:51:49.453295+0000","last_peered":"2024-08-18T13:51:49.453295+0000","last_clean":"2024-08-18T13:51:49.453295+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:51:49.453295+0000","last_undegraded":"2024-08-18T13:51:49.453295+0000","last_fullsized":"2024-08-18T13:51:49.453295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084356780000000003,"stat_sum":{"num_bytes":50488256,"num_objects":306,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":1250,"num_read_kb":43248,"num_write":629,"num_write_kb":49603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'84","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779153+0000","last_change":"2024-08-18T13:43:55.551442+0000","last_active":"2024-08-18T13:51:51.779153+0000","last_peered":"2024-08-18T13:51:51.779153+0000","last_clean":"2024-08-18T13:51:51.779153+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:51:51.779153+0000","last_undegraded":"2024-08-18T13:51:51.779153+0000","last_fullsized":"2024-08-18T13:51:51.779153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065977270000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"54'232","reported_seq":305,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.581418+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:51:41.581418+0000","last_peered":"2024-08-18T13:51:41.581418+0000","last_clean":"2024-08-18T13:51:41.581418+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:51:41.581418+0000","last_undegraded":"2024-08-18T13:51:41.581418+0000","last_fullsized":"2024-08-18T13:51:41.581418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583572,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":53278,"num_write":225,"num_write_kb":12431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"54'671","reported_seq":3262,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.458342+0000","last_change":"2024-08-18T13:43:56.142175+0000","last_active":"2024-08-18T13:51:49.458342+0000","last_peered":"2024-08-18T13:51:49.458342+0000","last_clean":"2024-08-18T13:51:49.458342+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:51:49.458342+0000","last_undegraded":"2024-08-18T13:51:49.458342+0000","last_fullsized":"2024-08-18T13:51:49.458342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076714670000000004,"stat_sum":{"num_bytes":16086174,"num_objects":343,"num_object_clones":0,"num_object_copies":1372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":529,"num_read_kb":12373,"num_write":668,"num_write_kb":24206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"54'86","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783436+0000","last_change":"2024-08-18T13:43:55.559262+0000","last_active":"2024-08-18T13:51:51.783436+0000","last_peered":"2024-08-18T13:51:51.783436+0000","last_clean":"2024-08-18T13:51:51.783436+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:51:51.783436+0000","last_undegraded":"2024-08-18T13:51:51.783436+0000","last_fullsized":"2024-08-18T13:51:51.783436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039212459999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"53'169","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.276349+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:50:57.276349+0000","last_peered":"2024-08-18T13:50:57.276349+0000","last_clean":"2024-08-18T13:50:57.276349+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:50:57.276349+0000","last_undegraded":"2024-08-18T13:50:57.276349+0000","last_fullsized":"2024-08-18T13:50:57.276349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8216,"num_write":168,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'644","reported_seq":3278,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.432661+0000","last_change":"2024-08-18T13:43:55.559022+0000","last_active":"2024-08-18T13:51:49.432661+0000","last_peered":"2024-08-18T13:51:49.432661+0000","last_clean":"2024-08-18T13:51:49.432661+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:51:49.432661+0000","last_undegraded":"2024-08-18T13:51:49.432661+0000","last_fullsized":"2024-08-18T13:51:49.432661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059375670000000004,"stat_sum":{"num_bytes":15740586,"num_objects":325,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":572,"num_read_kb":18359,"num_write":643,"num_write_kb":23895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"54'89","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778006+0000","last_change":"2024-08-18T13:43:56.153537+0000","last_active":"2024-08-18T13:51:51.778006+0000","last_peered":"2024-08-18T13:51:51.778006+0000","last_clean":"2024-08-18T13:51:51.778006+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:51:51.778006+0000","last_undegraded":"2024-08-18T13:51:51.778006+0000","last_fullsized":"2024-08-18T13:51:51.778006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057226309999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"54'233","reported_seq":282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.891732+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:50:56.891732+0000","last_peered":"2024-08-18T13:50:56.891732+0000","last_clean":"2024-08-18T13:50:56.891732+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:50:56.891732+0000","last_undegraded":"2024-08-18T13:50:56.891732+0000","last_fullsized":"2024-08-18T13:50:56.891732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":20397,"num_write":232,"num_write_kb":4254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"54'677","reported_seq":3169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.442883+0000","last_change":"2024-08-18T13:43:55.553346+0000","last_active":"2024-08-18T13:51:49.442883+0000","last_peered":"2024-08-18T13:51:49.442883+0000","last_clean":"2024-08-18T13:51:49.442883+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:51:49.442883+0000","last_undegraded":"2024-08-18T13:51:49.442883+0000","last_fullsized":"2024-08-18T13:51:49.442883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073620359999999998,"stat_sum":{"num_bytes":31240743,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":431,"num_read_kb":13577,"num_write":670,"num_write_kb":34133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"54'106","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778915+0000","last_change":"2024-08-18T13:43:55.861465+0000","last_active":"2024-08-18T13:51:51.778915+0000","last_peered":"2024-08-18T13:51:51.778915+0000","last_clean":"2024-08-18T13:51:51.778915+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:51:51.778915+0000","last_undegraded":"2024-08-18T13:51:51.778915+0000","last_fullsized":"2024-08-18T13:51:51.778915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059613369999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"54'9","reported_seq":51,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.581557+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:51:41.581557+0000","last_peered":"2024-08-18T13:51:41.581557+0000","last_clean":"2024-08-18T13:51:41.581557+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:51:41.581557+0000","last_undegraded":"2024-08-18T13:51:41.581557+0000","last_fullsized":"2024-08-18T13:51:41.581557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"54'699","reported_seq":3480,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.480893+0000","last_change":"2024-08-18T13:43:55.553902+0000","last_active":"2024-08-18T13:51:49.480893+0000","last_peered":"2024-08-18T13:51:49.480893+0000","last_clean":"2024-08-18T13:51:49.480893+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:51:49.480893+0000","last_undegraded":"2024-08-18T13:51:49.480893+0000","last_fullsized":"2024-08-18T13:51:49.480893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076808309999999999,"stat_sum":{"num_bytes":29636158,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":719,"num_read_kb":24704,"num_write":698,"num_write_kb":38783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"54'650","reported_seq":3292,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.034549+0000","last_change":"2024-08-18T13:43:56.152924+0000","last_active":"2024-08-18T13:51:49.034549+0000","last_peered":"2024-08-18T13:51:49.034549+0000","last_clean":"2024-08-18T13:51:49.034549+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:51:49.034549+0000","last_undegraded":"2024-08-18T13:51:49.034549+0000","last_fullsized":"2024-08-18T13:51:49.034549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068864499999999997,"stat_sum":{"num_bytes":24960867,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":580,"num_read_kb":20952,"num_write":650,"num_write_kb":24715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"54'91","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761698+0000","last_change":"2024-08-18T13:43:56.139883+0000","last_active":"2024-08-18T13:51:51.761698+0000","last_peered":"2024-08-18T13:51:51.761698+0000","last_clean":"2024-08-18T13:51:51.761698+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:51:51.761698+0000","last_undegraded":"2024-08-18T13:51:51.761698+0000","last_fullsized":"2024-08-18T13:51:51.761698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066095759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"54'199","reported_seq":252,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:42.157051+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:51:42.157051+0000","last_peered":"2024-08-18T13:51:42.157051+0000","last_clean":"2024-08-18T13:51:42.157051+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:51:42.157051+0000","last_undegraded":"2024-08-18T13:51:42.157051+0000","last_fullsized":"2024-08-18T13:51:42.157051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":197,"num_write_kb":12441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"54'658","reported_seq":3187,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.395492+0000","last_change":"2024-08-18T13:43:55.958984+0000","last_active":"2024-08-18T13:51:49.395492+0000","last_peered":"2024-08-18T13:51:49.395492+0000","last_clean":"2024-08-18T13:51:49.395492+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:51:49.395492+0000","last_undegraded":"2024-08-18T13:51:49.395492+0000","last_fullsized":"2024-08-18T13:51:49.395492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047613839999999996,"stat_sum":{"num_bytes":20867535,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":467,"num_read_kb":16390,"num_write":656,"num_write_kb":20726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764935+0000","last_change":"2024-08-18T13:43:55.552907+0000","last_active":"2024-08-18T13:51:51.764935+0000","last_peered":"2024-08-18T13:51:51.764935+0000","last_clean":"2024-08-18T13:51:51.764935+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:51:51.764935+0000","last_undegraded":"2024-08-18T13:51:51.764935+0000","last_fullsized":"2024-08-18T13:51:51.764935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068339799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"53'103","reported_seq":150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.907236+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:50:51.907236+0000","last_peered":"2024-08-18T13:50:51.907236+0000","last_clean":"2024-08-18T13:50:51.907236+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:50:51.907236+0000","last_undegraded":"2024-08-18T13:50:51.907236+0000","last_fullsized":"2024-08-18T13:50:51.907236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":103,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"54'131","reported_seq":186,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:34.672086+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:51:34.672086+0000","last_peered":"2024-08-18T13:51:34.672086+0000","last_clean":"2024-08-18T13:51:34.672086+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:51:34.672086+0000","last_undegraded":"2024-08-18T13:51:34.672086+0000","last_fullsized":"2024-08-18T13:51:34.672086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":128,"num_write_kb":12417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"54'91","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.768070+0000","last_change":"2024-08-18T13:43:56.154716+0000","last_active":"2024-08-18T13:51:51.768070+0000","last_peered":"2024-08-18T13:51:51.768070+0000","last_clean":"2024-08-18T13:51:51.768070+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:51:51.768070+0000","last_undegraded":"2024-08-18T13:51:51.768070+0000","last_fullsized":"2024-08-18T13:51:51.768070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053680339999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"54'664","reported_seq":3634,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.568805+0000","last_change":"2024-08-18T13:43:55.917879+0000","last_active":"2024-08-18T13:51:49.568805+0000","last_peered":"2024-08-18T13:51:49.568805+0000","last_clean":"2024-08-18T13:51:49.568805+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:51:49.568805+0000","last_undegraded":"2024-08-18T13:51:49.568805+0000","last_fullsized":"2024-08-18T13:51:49.568805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035122529999999999,"stat_sum":{"num_bytes":30328916,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":908,"num_read_kb":29115,"num_write":663,"num_write_kb":34071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"54'14","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.993056+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:51:38.993056+0000","last_peered":"2024-08-18T13:51:38.993056+0000","last_clean":"2024-08-18T13:51:38.993056+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:51:38.993056+0000","last_undegraded":"2024-08-18T13:51:38.993056+0000","last_fullsized":"2024-08-18T13:51:38.993056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":19,"num_write_kb":4249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757533+0000","last_change":"2024-08-18T13:43:55.555770+0000","last_active":"2024-08-18T13:51:51.757533+0000","last_peered":"2024-08-18T13:51:51.757533+0000","last_clean":"2024-08-18T13:51:51.757533+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:51:51.757533+0000","last_undegraded":"2024-08-18T13:51:51.757533+0000","last_fullsized":"2024-08-18T13:51:51.757533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072935329999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"54'656","reported_seq":3234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.680798+0000","last_change":"2024-08-18T13:43:55.555171+0000","last_active":"2024-08-18T13:51:49.680798+0000","last_peered":"2024-08-18T13:51:49.680798+0000","last_clean":"2024-08-18T13:51:49.680798+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:51:49.680798+0000","last_undegraded":"2024-08-18T13:51:49.680798+0000","last_fullsized":"2024-08-18T13:51:49.680798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063112100000000003,"stat_sum":{"num_bytes":21201155,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":516,"num_read_kb":24278,"num_write":657,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"54'107","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764519+0000","last_change":"2024-08-18T13:43:55.959563+0000","last_active":"2024-08-18T13:51:51.764519+0000","last_peered":"2024-08-18T13:51:51.764519+0000","last_clean":"2024-08-18T13:51:51.764519+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:51:51.764519+0000","last_undegraded":"2024-08-18T13:51:51.764519+0000","last_fullsized":"2024-08-18T13:51:51.764519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045422170000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"54'110","reported_seq":2173,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776910+0000","last_change":"2024-08-18T13:43:55.919976+0000","last_active":"2024-08-18T13:51:51.776910+0000","last_peered":"2024-08-18T13:51:51.776910+0000","last_clean":"2024-08-18T13:51:51.776910+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:51:51.776910+0000","last_undegraded":"2024-08-18T13:51:51.776910+0000","last_fullsized":"2024-08-18T13:51:51.776910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032950929999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"54'101","reported_seq":2174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757889+0000","last_change":"2024-08-18T13:43:55.863304+0000","last_active":"2024-08-18T13:51:51.757889+0000","last_peered":"2024-08-18T13:51:51.757889+0000","last_clean":"2024-08-18T13:51:51.757889+0000","last_became_active":"2024-08-18T13:40:16.610568+0000","last_became_peered":"2024-08-18T13:40:16.610568+0000","last_unstale":"2024-08-18T13:51:51.757889+0000","last_undegraded":"2024-08-18T13:51:51.757889+0000","last_fullsized":"2024-08-18T13:51:51.757889+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:53:53.687401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066503589999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"54'99","reported_seq":2163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766416+0000","last_change":"2024-08-18T13:43:56.153594+0000","last_active":"2024-08-18T13:51:51.766416+0000","last_peered":"2024-08-18T13:51:51.766416+0000","last_clean":"2024-08-18T13:51:51.766416+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:51:51.766416+0000","last_undegraded":"2024-08-18T13:51:51.766416+0000","last_fullsized":"2024-08-18T13:51:51.766416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056450420000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"54'80","reported_seq":2143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764106+0000","last_change":"2024-08-18T13:43:55.554865+0000","last_active":"2024-08-18T13:51:51.764106+0000","last_peered":"2024-08-18T13:51:51.764106+0000","last_clean":"2024-08-18T13:51:51.764106+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:51:51.764106+0000","last_undegraded":"2024-08-18T13:51:51.764106+0000","last_fullsized":"2024-08-18T13:51:51.764106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068042459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"54'74","reported_seq":2138,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761397+0000","last_change":"2024-08-18T13:43:56.142479+0000","last_active":"2024-08-18T13:51:51.761397+0000","last_peered":"2024-08-18T13:51:51.761397+0000","last_clean":"2024-08-18T13:51:51.761397+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:51:51.761397+0000","last_undegraded":"2024-08-18T13:51:51.761397+0000","last_fullsized":"2024-08-18T13:51:51.761397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079157710000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779808+0000","last_change":"2024-08-18T13:43:55.959839+0000","last_active":"2024-08-18T13:51:51.779808+0000","last_peered":"2024-08-18T13:51:51.779808+0000","last_clean":"2024-08-18T13:51:51.779808+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:51:51.779808+0000","last_undegraded":"2024-08-18T13:51:51.779808+0000","last_fullsized":"2024-08-18T13:51:51.779808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029546540000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"54'85","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755227+0000","last_change":"2024-08-18T13:43:55.555415+0000","last_active":"2024-08-18T13:51:51.755227+0000","last_peered":"2024-08-18T13:51:51.755227+0000","last_clean":"2024-08-18T13:51:51.755227+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:51:51.755227+0000","last_undegraded":"2024-08-18T13:51:51.755227+0000","last_fullsized":"2024-08-18T13:51:51.755227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055234810000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"54'92","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762023+0000","last_change":"2024-08-18T13:43:55.916616+0000","last_active":"2024-08-18T13:51:51.762023+0000","last_peered":"2024-08-18T13:51:51.762023+0000","last_clean":"2024-08-18T13:51:51.762023+0000","last_became_active":"2024-08-18T13:40:16.614889+0000","last_became_peered":"2024-08-18T13:40:16.614889+0000","last_unstale":"2024-08-18T13:51:51.762023+0000","last_undegraded":"2024-08-18T13:51:51.762023+0000","last_fullsized":"2024-08-18T13:51:51.762023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:37:46.326075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033397829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761272+0000","last_change":"2024-08-18T13:43:55.554199+0000","last_active":"2024-08-18T13:51:51.761272+0000","last_peered":"2024-08-18T13:51:51.761272+0000","last_clean":"2024-08-18T13:51:51.761272+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:51:51.761272+0000","last_undegraded":"2024-08-18T13:51:51.761272+0000","last_fullsized":"2024-08-18T13:51:51.761272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070000389999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"54'95","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.780212+0000","last_change":"2024-08-18T13:43:55.863255+0000","last_active":"2024-08-18T13:51:51.780212+0000","last_peered":"2024-08-18T13:51:51.780212+0000","last_clean":"2024-08-18T13:51:51.780212+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:51:51.780212+0000","last_undegraded":"2024-08-18T13:51:51.780212+0000","last_fullsized":"2024-08-18T13:51:51.780212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070451519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"54'88","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778638+0000","last_change":"2024-08-18T13:43:56.152879+0000","last_active":"2024-08-18T13:51:51.778638+0000","last_peered":"2024-08-18T13:51:51.778638+0000","last_clean":"2024-08-18T13:51:51.778638+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:51:51.778638+0000","last_undegraded":"2024-08-18T13:51:51.778638+0000","last_fullsized":"2024-08-18T13:51:51.778638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070254549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.759908+0000","last_change":"2024-08-18T13:43:55.956991+0000","last_active":"2024-08-18T13:51:51.759908+0000","last_peered":"2024-08-18T13:51:51.759908+0000","last_clean":"2024-08-18T13:51:51.759908+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:51:51.759908+0000","last_undegraded":"2024-08-18T13:51:51.759908+0000","last_fullsized":"2024-08-18T13:51:51.759908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003261548,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"54'85","reported_seq":2143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758255+0000","last_change":"2024-08-18T13:43:56.141939+0000","last_active":"2024-08-18T13:51:51.758255+0000","last_peered":"2024-08-18T13:51:51.758255+0000","last_clean":"2024-08-18T13:51:51.758255+0000","last_became_active":"2024-08-18T13:40:16.612760+0000","last_became_peered":"2024-08-18T13:40:16.612760+0000","last_unstale":"2024-08-18T13:51:51.758255+0000","last_undegraded":"2024-08-18T13:51:51.758255+0000","last_fullsized":"2024-08-18T13:51:51.758255+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:58:42.688761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076223990000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767420+0000","last_change":"2024-08-18T13:43:55.917355+0000","last_active":"2024-08-18T13:51:51.767420+0000","last_peered":"2024-08-18T13:51:51.767420+0000","last_clean":"2024-08-18T13:51:51.767420+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:51:51.767420+0000","last_undegraded":"2024-08-18T13:51:51.767420+0000","last_fullsized":"2024-08-18T13:51:51.767420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027557649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"54'94","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779547+0000","last_change":"2024-08-18T13:43:55.860909+0000","last_active":"2024-08-18T13:51:51.779547+0000","last_peered":"2024-08-18T13:51:51.779547+0000","last_clean":"2024-08-18T13:51:51.779547+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:51:51.779547+0000","last_undegraded":"2024-08-18T13:51:51.779547+0000","last_fullsized":"2024-08-18T13:51:51.779547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069473399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"54'96","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775325+0000","last_change":"2024-08-18T13:43:55.860127+0000","last_active":"2024-08-18T13:51:51.775325+0000","last_peered":"2024-08-18T13:51:51.775325+0000","last_clean":"2024-08-18T13:51:51.775325+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:51:51.775325+0000","last_undegraded":"2024-08-18T13:51:51.775325+0000","last_fullsized":"2024-08-18T13:51:51.775325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070196240000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761102+0000","last_change":"2024-08-18T13:43:56.136762+0000","last_active":"2024-08-18T13:51:51.761102+0000","last_peered":"2024-08-18T13:51:51.761102+0000","last_clean":"2024-08-18T13:51:51.761102+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:51:51.761102+0000","last_undegraded":"2024-08-18T13:51:51.761102+0000","last_fullsized":"2024-08-18T13:51:51.761102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004509931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.760407+0000","last_change":"2024-08-18T13:43:56.154301+0000","last_active":"2024-08-18T13:51:51.760407+0000","last_peered":"2024-08-18T13:51:51.760407+0000","last_clean":"2024-08-18T13:51:51.760407+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:51:51.760407+0000","last_undegraded":"2024-08-18T13:51:51.760407+0000","last_fullsized":"2024-08-18T13:51:51.760407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062343609999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"54'81","reported_seq":2144,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767251+0000","last_change":"2024-08-18T13:43:55.554507+0000","last_active":"2024-08-18T13:51:51.767251+0000","last_peered":"2024-08-18T13:51:51.767251+0000","last_clean":"2024-08-18T13:51:51.767251+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:51:51.767251+0000","last_undegraded":"2024-08-18T13:51:51.767251+0000","last_fullsized":"2024-08-18T13:51:51.767251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072121939999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"54'77","reported_seq":2141,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767294+0000","last_change":"2024-08-18T13:43:55.558437+0000","last_active":"2024-08-18T13:51:51.767294+0000","last_peered":"2024-08-18T13:51:51.767294+0000","last_clean":"2024-08-18T13:51:51.767294+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:51:51.767294+0000","last_undegraded":"2024-08-18T13:51:51.767294+0000","last_fullsized":"2024-08-18T13:51:51.767294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057680220000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776234+0000","last_change":"2024-08-18T13:43:55.958601+0000","last_active":"2024-08-18T13:51:51.776234+0000","last_peered":"2024-08-18T13:51:51.776234+0000","last_clean":"2024-08-18T13:51:51.776234+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:51:51.776234+0000","last_undegraded":"2024-08-18T13:51:51.776234+0000","last_fullsized":"2024-08-18T13:51:51.776234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049025989999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779265+0000","last_change":"2024-08-18T13:43:55.959710+0000","last_active":"2024-08-18T13:51:51.779265+0000","last_peered":"2024-08-18T13:51:51.779265+0000","last_clean":"2024-08-18T13:51:51.779265+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:51:51.779265+0000","last_undegraded":"2024-08-18T13:51:51.779265+0000","last_fullsized":"2024-08-18T13:51:51.779265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029569129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"54'83","reported_seq":2146,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781497+0000","last_change":"2024-08-18T13:43:55.552920+0000","last_active":"2024-08-18T13:51:51.781497+0000","last_peered":"2024-08-18T13:51:51.781497+0000","last_clean":"2024-08-18T13:51:51.781497+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:51:51.781497+0000","last_undegraded":"2024-08-18T13:51:51.781497+0000","last_fullsized":"2024-08-18T13:51:51.781497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062586680000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"54'74","reported_seq":2137,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777962+0000","last_change":"2024-08-18T13:43:55.551745+0000","last_active":"2024-08-18T13:51:51.777962+0000","last_peered":"2024-08-18T13:51:51.777962+0000","last_clean":"2024-08-18T13:51:51.777962+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:51:51.777962+0000","last_undegraded":"2024-08-18T13:51:51.777962+0000","last_fullsized":"2024-08-18T13:51:51.777962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064056340000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"54'105","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774468+0000","last_change":"2024-08-18T13:43:55.554037+0000","last_active":"2024-08-18T13:51:51.774468+0000","last_peered":"2024-08-18T13:51:51.774468+0000","last_clean":"2024-08-18T13:51:51.774468+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:51:51.774468+0000","last_undegraded":"2024-08-18T13:51:51.774468+0000","last_fullsized":"2024-08-18T13:51:51.774468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050911749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"54'93","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774885+0000","last_change":"2024-08-18T13:43:55.862998+0000","last_active":"2024-08-18T13:51:51.774885+0000","last_peered":"2024-08-18T13:51:51.774885+0000","last_clean":"2024-08-18T13:51:51.774885+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:51:51.774885+0000","last_undegraded":"2024-08-18T13:51:51.774885+0000","last_fullsized":"2024-08-18T13:51:51.774885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068654359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"54'102","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776207+0000","last_change":"2024-08-18T13:43:55.550186+0000","last_active":"2024-08-18T13:51:51.776207+0000","last_peered":"2024-08-18T13:51:51.776207+0000","last_clean":"2024-08-18T13:51:51.776207+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:51:51.776207+0000","last_undegraded":"2024-08-18T13:51:51.776207+0000","last_fullsized":"2024-08-18T13:51:51.776207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055002549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"54'76","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774945+0000","last_change":"2024-08-18T13:43:55.859954+0000","last_active":"2024-08-18T13:51:51.774945+0000","last_peered":"2024-08-18T13:51:51.774945+0000","last_clean":"2024-08-18T13:51:51.774945+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:51:51.774945+0000","last_undegraded":"2024-08-18T13:51:51.774945+0000","last_fullsized":"2024-08-18T13:51:51.774945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069338170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762078+0000","last_change":"2024-08-18T13:43:56.150131+0000","last_active":"2024-08-18T13:51:51.762078+0000","last_peered":"2024-08-18T13:51:51.762078+0000","last_clean":"2024-08-18T13:51:51.762078+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:51:51.762078+0000","last_undegraded":"2024-08-18T13:51:51.762078+0000","last_fullsized":"2024-08-18T13:51:51.762078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044622389999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"54'76","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763798+0000","last_change":"2024-08-18T13:43:56.136824+0000","last_active":"2024-08-18T13:51:51.763798+0000","last_peered":"2024-08-18T13:51:51.763798+0000","last_clean":"2024-08-18T13:51:51.763798+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:51:51.763798+0000","last_undegraded":"2024-08-18T13:51:51.763798+0000","last_fullsized":"2024-08-18T13:51:51.763798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035965799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"54'86","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778822+0000","last_change":"2024-08-18T13:43:55.552484+0000","last_active":"2024-08-18T13:51:51.778822+0000","last_peered":"2024-08-18T13:51:51.778822+0000","last_clean":"2024-08-18T13:51:51.778822+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:51:51.778822+0000","last_undegraded":"2024-08-18T13:51:51.778822+0000","last_fullsized":"2024-08-18T13:51:51.778822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067690759999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":938443088,"num_objects":10906,"num_object_clones":0,"num_object_copies":43624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10906,"num_whiteouts":0,"num_read":21895,"num_read_kb":705026,"num_write":21484,"num_write_kb":1061752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2001649664,"data_stored":2001649664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5769,"ondisk_log_size":5769,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":184495130,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":586,"num_read_kb":685208,"num_write":6404,"num_write_kb":185940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":369049600,"data_stored":368990260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6329,"ondisk_log_size":6329,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599760,"kb_used_data":294088,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93772080,"statfs":{"total":96636764160,"available":96022609920,"internally_reserved":0,"allocated":301146112,"data_stored":300847166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2999999999999998,"5min":1.026,"15min":0.878},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":25.744,"5min":25.744,"15min":25.744},"last":1.254},{"interface":"front","average":{"1min":1.3149999999999999,"5min":0.84999999999999998,"15min":0.82599999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.33100000000000002},"max":{"1min":8.1679999999999993,"5min":8.1679999999999993,"15min":8.1679999999999993},"last":0.93799999999999994}]},{"osd":1,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":0.90700000000000003,"15min":0.98599999999999999},"min":{"1min":0.5,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":10.151999999999999,"5min":10.151999999999999,"15min":39.203000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":2.323,"5min":1.111,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":25.852,"5min":25.852,"15min":25.852},"last":0.96499999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":1.121,"15min":0.95099999999999996},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.39400000000000002},"max":{"1min":25.818999999999999,"5min":25.818999999999999,"15min":25.818999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.85199999999999998},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.35799999999999998},"max":{"1min":1.133,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.0840000000000001,"15min":1.135},"min":{"1min":0.64700000000000002,"5min":0.42699999999999999,"15min":0.377},"max":{"1min":17.300999999999998,"5min":17.300999999999998,"15min":42.905000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":2.4689999999999999,"5min":1.238,"15min":1.115},"min":{"1min":0.75600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":25.786999999999999,"5min":25.786999999999999,"15min":25.786999999999999},"last":1.323}]},{"osd":4,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.29699999999999999},"max":{"1min":8.3870000000000005,"5min":8.3870000000000005,"15min":8.3870000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.171,"15min":1.23},"min":{"1min":0.79800000000000004,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":17.056999999999999,"5min":17.056999999999999,"15min":52.673000000000002},"last":1.127}]},{"osd":5,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":1.274,"15min":1.119},"min":{"1min":0.44,"5min":0.44,"15min":0.33700000000000002},"max":{"1min":25.881,"5min":25.881,"15min":25.881},"last":1.087},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.1080000000000001,"15min":1.073},"min":{"1min":0.73899999999999999,"5min":0.54200000000000004,"15min":0.37},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":0.77100000000000002}]},{"osd":6,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.1659999999999999,"15min":1.1259999999999999},"min":{"1min":0.83799999999999997,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":8.0510000000000002,"5min":8.0510000000000002,"15min":8.0510000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.1990000000000001,"15min":1.26},"min":{"1min":0.85499999999999998,"5min":0.49399999999999999,"15min":0.435},"max":{"1min":10.523,"5min":10.523,"15min":39.161000000000001},"last":1.02}]}]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610072,"kb_used_data":312720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93761768,"statfs":{"total":96636764160,"available":96012050432,"internally_reserved":0,"allocated":320225280,"data_stored":319926507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76500000000000001,"15min":0.88},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.165,"5min":1.21,"15min":4.532},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.248},"max":{"1min":1.258,"5min":1.3200000000000001,"15min":1.9650000000000001},"last":0.73899999999999999}]},{"osd":1,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.92000000000000004},"min":{"1min":0.42399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2749999999999999,"5min":1.294,"15min":4.468},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.91900000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.30499999999999999},"max":{"1min":1.1990000000000001,"5min":1.286,"15min":2.3620000000000001},"last":1.01}]},{"osd":2,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77700000000000002,"15min":0.83899999999999997},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.28000000000000003},"max":{"1min":1.085,"5min":1.177,"15min":1.413},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.97299999999999998},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1060000000000001,"5min":1.468,"15min":4.5},"last":0.69999999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88700000000000001,"15min":0.94199999999999995},"min":{"1min":0.60899999999999999,"5min":0.41999999999999998,"15min":0.35399999999999998},"max":{"1min":1.1919999999999999,"5min":1.276,"15min":4.3250000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.91600000000000004},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.32900000000000001},"max":{"1min":1.0920000000000001,"5min":1.306,"15min":1.8069999999999999},"last":0.94899999999999995}]},{"osd":4,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92700000000000005,"15min":1.0109999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.228,"5min":1.552,"15min":5.2549999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.7230000000000001,"15min":4.6600000000000001},"last":1.145}]},{"osd":5,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":1.0129999999999999},"min":{"1min":0.54900000000000004,"5min":0.45400000000000001,"15min":0.373},"max":{"1min":1.179,"5min":1.3160000000000001,"15min":4.4009999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.042,"15min":1.1499999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.272,"5min":1.6879999999999999,"15min":6.1260000000000003},"last":1.125}]},{"osd":7,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0669999999999999,"15min":1.111},"min":{"1min":0.67600000000000005,"5min":0.55500000000000005,"15min":0.40500000000000003},"max":{"1min":1.3009999999999999,"5min":1.4830000000000001,"15min":1.9299999999999999},"last":1.103},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0609999999999999,"15min":1.1080000000000001},"min":{"1min":0.67500000000000004,"5min":0.46300000000000002,"15min":0.40200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":2.1150000000000002},"last":1.2330000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509594,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570040,"kb_used_data":272688,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93801800,"statfs":{"total":96636764160,"available":96053043200,"internally_reserved":0,"allocated":279232512,"data_stored":278928482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.24299999999999999},"max":{"1min":1.1679999999999999,"5min":1.204,"15min":1.22},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":1.085,"5min":1.339,"15min":1.73},"last":0.84399999999999997}]},{"osd":1,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82499999999999996,"15min":0.86799999999999999},"min":{"1min":0.54100000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.2909999999999999,"5min":1.415,"15min":3.6299999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.36699999999999999,"15min":0.27600000000000002},"max":{"1min":1.0589999999999999,"5min":1.28,"15min":1.5169999999999999},"last":1.139}]},{"osd":2,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.85799999999999998},"min":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.5920000000000001},"last":0.77000000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92100000000000004,"15min":0.91800000000000004},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.309},"max":{"1min":1.111,"5min":1.387,"15min":1.45},"last":1.222},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.641,"15min":1.641},"last":1.111}]},{"osd":5,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.313},"max":{"1min":1.3640000000000001,"5min":1.575,"15min":3.7789999999999999},"last":1.081},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.998,"15min":1.0149999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3129999999999999,"5min":1.754,"15min":1.754},"last":0.99399999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0580000000000001,"15min":1.079},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.504},"max":{"1min":1.6259999999999999,"5min":1.901,"15min":7.8490000000000002},"last":1.026},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.016,"15min":1.0629999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2809999999999999,"5min":1.5229999999999999,"15min":6.3380000000000001},"last":1.282}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40000000000000002},"max":{"1min":1.2470000000000001,"5min":1.7470000000000001,"15min":1.8400000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.151,"15min":1.1819999999999999},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.42399999999999999},"max":{"1min":1.6850000000000001,"5min":2.4569999999999999,"15min":3.7410000000000001},"last":1.165}]}]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683436,"kb_used_data":373604,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93688404,"statfs":{"total":96636764160,"available":95936925696,"internally_reserved":0,"allocated":382570496,"data_stored":382269906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.29499999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.89500000000000002,"5min":1.2050000000000001,"15min":1.2170000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67800000000000005,"15min":0.69299999999999995},"min":{"1min":0.44900000000000001,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":1.413,"15min":1.413},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.68600000000000005},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86499999999999999,"5min":1.46,"15min":3.4820000000000002},"last":1.089},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.751},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.8979999999999999},"last":0.96199999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.48599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1910000000000001,"5min":1.6419999999999999,"15min":2.415},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.87,"15min":0.86899999999999999},"min":{"1min":0.57799999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.2589999999999999,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.0580000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.173,"5min":1.681,"15min":3.2490000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.052},"min":{"1min":0.66300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5189999999999999,"5min":2.4870000000000001,"15min":3.8799999999999999},"last":1.2270000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.996,"15min":4.1059999999999999},"last":1.119},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0089999999999999,"15min":1.05},"min":{"1min":0.63200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1719999999999999,"5min":1.627,"15min":3.2799999999999998},"last":1.1459999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.504,"5min":0.504,"15min":0.41999999999999998},"max":{"1min":1.3480000000000001,"5min":1.833,"15min":2.1080000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.054,"15min":1.0880000000000001},"min":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.573,"15min":2.7469999999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4419999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.169},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.087,"15min":1.0960000000000001},"min":{"1min":0.71499999999999997,"5min":0.63200000000000001,"15min":0.51100000000000001},"max":{"1min":1.583,"5min":1.9039999999999999,"15min":2.1429999999999998},"last":1.337}]}]},{"osd":2,"up_from":14,"seq":60129542298,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530780,"kb_used_data":229268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93841060,"statfs":{"total":96636764160,"available":96093245440,"internally_reserved":0,"allocated":234770432,"data_stored":234477374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.186},"max":{"1min":1.0369999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":1.0960000000000001,"5min":3.27,"15min":3.27},"last":0.5}]},{"osd":1,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.038,"5min":1.1040000000000001,"15min":1.696},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":3.2290000000000001,"15min":3.2290000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.78200000000000003},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.253},"max":{"1min":0.995,"5min":1.077,"15min":12.468999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.84399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35899999999999999,"15min":0.27500000000000002},"max":{"1min":1.077,"5min":1.3540000000000001,"15min":1.744},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2709999999999999,"5min":1.681,"15min":1.8480000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.94799999999999995},"min":{"1min":0.46899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2609999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1759999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39000000000000001},"max":{"1min":1.149,"5min":1.498,"15min":1.627},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96899999999999997,"15min":0.98799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.42199999999999999},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.649},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.0309999999999999,"15min":1.0649999999999999},"min":{"1min":0.67000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":1.5820000000000001,"15min":1.905},"last":0.878}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0549999999999999,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.60399999999999998,"15min":0.436},"max":{"1min":1.3140000000000001,"5min":1.5089999999999999,"15min":1.9490000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.0149999999999999},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.46400000000000002},"max":{"1min":1.343,"5min":1.4379999999999999,"15min":1.7669999999999999},"last":0.90200000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542298,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677976,"kb_used_data":376464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93693864,"statfs":{"total":96636764160,"available":95942516736,"internally_reserved":0,"allocated":385499136,"data_stored":385199863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.65200000000000002},"min":{"1min":0.28899999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.94899999999999995,"5min":1.3009999999999999,"15min":3.1160000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.72099999999999997},"min":{"1min":0.30599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.107,"5min":1.327,"15min":3.718},"last":0.76900000000000002}]},{"osd":2,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0549999999999999,"5min":4.968,"15min":4.968},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33400000000000002,"15min":0.20899999999999999},"max":{"1min":1.327,"5min":1.3560000000000001,"15min":9.3309999999999995},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4810000000000001,"15min":1.927},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79400000000000004,"15min":0.84899999999999998},"min":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":1.46,"5min":1.46,"15min":6.8559999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.28000000000000003},"max":{"1min":1.488,"5min":5.1020000000000003,"15min":5.6529999999999996},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.63400000000000001,"5min":0.39700000000000002,"15min":0.28599999999999998},"max":{"1min":1.4410000000000001,"5min":1.554,"15min":1.5740000000000001},"last":1.125}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3859999999999999,"5min":1.546,"15min":3.7570000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1,"5min":1.04,"15min":1.1040000000000001},"min":{"1min":0.47799999999999998,"5min":0.46400000000000002,"15min":0.45700000000000002},"max":{"1min":1.591,"5min":5.1799999999999997,"15min":19.027999999999999},"last":1.272}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.40699999999999997},"max":{"1min":1.5529999999999999,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.113,"5min":1.0880000000000001,"15min":1.131},"min":{"1min":0.54900000000000004,"5min":0.49099999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.861},"last":0.98799999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.069,"15min":1.119},"min":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58799999999999997},"max":{"1min":1.383,"5min":1.526,"15min":3.8100000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40000000000000002},"max":{"1min":1.431,"5min":5.3099999999999996,"15min":5.3099999999999996},"last":1.3220000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542298,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549780,"kb_used_data":248268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93822060,"statfs":{"total":96636764160,"available":96073789440,"internally_reserved":0,"allocated":254226432,"data_stored":253919977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.65400000000000003},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":5.3159999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.33500000000000002,"5min":0.254,"15min":0.19500000000000001},"max":{"1min":0.93600000000000005,"5min":1.2370000000000001,"15min":3.4809999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.47099999999999997,"5min":0.32600000000000001,"15min":0.26300000000000001},"max":{"1min":0.99199999999999999,"5min":1.222,"15min":3.3029999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.28},"last":0.90000000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.288,"5min":1.288,"15min":5.399},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86799999999999999,"15min":0.93100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.33900000000000002},"max":{"1min":1.1799999999999999,"5min":1.365,"15min":5.0359999999999996},"last":0.99099999999999999}]},{"osd":4,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.37},"max":{"1min":1.3899999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":8.7799999999999994},"last":0.93999999999999995}]},{"osd":5,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96099999999999997,"15min":0.97499999999999998},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.377},{"interface":"front","average":{"1min":1.087,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.40300000000000002},"max":{"1min":1.51,"5min":1.52,"15min":1.8879999999999999},"last":0.82499999999999996}]},{"osd":6,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.77800000000000002,"5min":0.46500000000000002,"15min":0.374},"max":{"1min":1.423,"5min":1.599,"15min":3.7200000000000002},"last":1.171},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.042,"15min":1.085},"min":{"1min":0.68600000000000005,"5min":0.58099999999999996,"15min":0.52600000000000002},"max":{"1min":1.55,"5min":1.659,"15min":3.2789999999999999},"last":1.4239999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.143,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.60599999999999998,"15min":0.40300000000000002},"max":{"1min":1.514,"5min":1.704,"15min":36.295000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.171,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.55300000000000005,"15min":0.42999999999999999},"max":{"1min":1.5740000000000001,"5min":1.706,"15min":3.448},"last":1.206}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515540,"kb_used_data":214028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93856300,"statfs":{"total":96636764160,"available":96108851200,"internally_reserved":0,"allocated":219164672,"data_stored":218869425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.81100000000000005},"min":{"1min":0.48199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.135,"5min":1.2789999999999999,"15min":1.901},"last":0.747},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.81000000000000005,"15min":0.85599999999999998},"min":{"1min":0.63900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":4.0810000000000004},"last":0.88100000000000001}]},{"osd":1,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.82099999999999995},"min":{"1min":0.51900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.101,"5min":1.429,"15min":2.4809999999999999},"last":1.022},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1799999999999999,"5min":1.456,"15min":1.7749999999999999},"last":0.93400000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.63200000000000001,"5min":0.46999999999999997,"15min":0.41799999999999998},"max":{"1min":1.1419999999999999,"5min":1.524,"15min":1.774},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.874,"15min":0.93899999999999995},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.32500000000000001},"max":{"1min":1.1299999999999999,"5min":1.3160000000000001,"15min":4.1559999999999997},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.94399999999999995,"15min":0.97999999999999998},"min":{"1min":0.59799999999999998,"5min":0.54600000000000004,"15min":0.36199999999999999},"max":{"1min":1.3300000000000001,"5min":1.4410000000000001,"15min":4.5510000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.92700000000000005,"15min":0.98499999999999999},"min":{"1min":0.72199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.423,"5min":1.4470000000000001,"15min":3.976},"last":1.143}]},{"osd":4,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.7909999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0660000000000001,"15min":1.1120000000000001},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.6839999999999999,"15min":2.532},"last":1.2430000000000001}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.03,"15min":1.085},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3109999999999999,"5min":1.532,"15min":4.2050000000000001},"last":1.071},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0569999999999999,"15min":1.113},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":2.911},"last":1.2150000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.089,"15min":1.165},"min":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2989999999999999,"5min":1.659,"15min":2.569},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.1359999999999999,"15min":1.361},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40400000000000003},"max":{"1min":1.407,"5min":1.605,"15min":58.290999999999997},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41955328,"data_stored":41943666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50331808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21549056,"data_stored":21547239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34140160,"data_stored":34130811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53907456,"data_stored":53894667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41947136,"data_stored":41943130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50332308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":74870784,"data_stored":74866631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211058688,"data_stored":211058688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334540800,"data_stored":334540800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212602880,"data_stored":212602880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347811840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":224706560,"data_stored":224706560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176599040,"data_stored":176599040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":269266944,"data_stored":269266944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":225062912,"data_stored":225062912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:51:57.966 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:51:58.071 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f9ce5700 1 Processor -- start 2024-08-18T13:51:58.072 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f9ce5700 1 -- start start 2024-08-18T13:51:58.073 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f9ce5700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa4f40c46f0 0x7fa4f4156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f9ce5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f4067d50 0x7fa4f4153980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f9ce5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f41534f0 con 0x7fa4f40c4320 2024-08-18T13:51:58.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f9ce5700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f41574c0 con 0x7fa4f40c46f0 2024-08-18T13:51:58.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f9ce5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f4152f70 con 0x7fa4f4067d50 2024-08-18T13:51:58.074 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4ebfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa4f40c46f0 0x7fa4f4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4ebfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fa4f40c46f0 0x7fa4f4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44920/0 (socket says 172.21.15.118:44920) 2024-08-18T13:51:58.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4ebfff700 1 -- 172.21.15.118:0/4259621124 learned_addr learned my addr 172.21.15.118:0/4259621124 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:58.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4f94e4700 1 --2- 172.21.15.118:0/4259621124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f4067d50 0x7fa4f4153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4ebfff700 1 -- 172.21.15.118:0/4259621124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f4067d50 msgr2=0x7fa4f4153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:58.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4ebfff700 1 --2- 172.21.15.118:0/4259621124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f4067d50 0x7fa4f4153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.066+0000 7fa4ebfff700 1 -- 172.21.15.118:0/4259621124 >> v1:172.21.15.178:6789/0 conn(0x7fa4f40c4320 legacy=0x7fa4f405e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:58.075 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4ebfff700 1 -- 172.21.15.118:0/4259621124 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f4156c20 con 0x7fa4f40c46f0 2024-08-18T13:51:58.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f94e4700 1 --2- 172.21.15.118:0/4259621124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f4067d50 0x7fa4f4153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:51:58.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4ebfff700 1 --2- 172.21.15.118:0/4259621124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f40c46f0 0x7fa4f4156570 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40c81b0 tx=0x7fa4e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14076bd196b950 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4eb7fe700 1 -- 172.21.15.118:0/4259621124 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400b820 con 0x7fa4f40c46f0 2024-08-18T13:51:58.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4eb7fe700 1 -- 172.21.15.118:0/4259621124 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e400b980 con 0x7fa4f40c46f0 2024-08-18T13:51:58.076 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4eb7fe700 1 -- 172.21.15.118:0/4259621124 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400bd20 con 0x7fa4f40c46f0 2024-08-18T13:51:58.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/4259621124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f40c46f0 msgr2=0x7fa4f4156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.077 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/4259621124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f40c46f0 0x7fa4f4156570 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40c81b0 tx=0x7fa4e4009250 comp rx=0 tx=0).stop 2024-08-18T13:51:58.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/4259621124 shutdown_connections 2024-08-18T13:51:58.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/4259621124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f4067d50 0x7fa4f4153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/4259621124 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f40c46f0 0x7fa4f4156570 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/4259621124 >> 172.21.15.118:0/4259621124 conn(0x7fa4f40c81e0 msgr2=0x7fa4f40c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:58.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/4259621124 shutdown_connections 2024-08-18T13:51:58.078 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.070+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/4259621124 wait complete. 2024-08-18T13:51:58.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 Processor -- start 2024-08-18T13:51:58.079 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 -- start start 2024-08-18T13:51:58.081 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f4067d50 0x7fa4f4151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.082 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f40c4320 0x7fa4f41426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.082 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40c46f0 0x7fa4f414b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.082 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f413d420 con 0x7fa4f4067d50 2024-08-18T13:51:58.082 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f41558e0 con 0x7fa4f40c4320 2024-08-18T13:51:58.082 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f4151190 con 0x7fa4f40c46f0 2024-08-18T13:51:58.082 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f4067d50 0x7fa4f4151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4ebfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f40c4320 0x7fa4f41426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f94e4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40c46f0 0x7fa4f414b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4ebfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f40c4320 0x7fa4f41426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:58552/0 (socket says 172.21.15.118:58552) 2024-08-18T13:51:58.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4ebfff700 1 -- 172.21.15.118:0/3306163449 learned_addr learned my addr 172.21.15.118:0/3306163449 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:58.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f4067d50 0x7fa4f4151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44936/0 (socket says 172.21.15.118:44936) 2024-08-18T13:51:58.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 -- 172.21.15.118:0/3306163449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40c46f0 msgr2=0x7fa4f414b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40c46f0 0x7fa4f414b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 -- 172.21.15.118:0/3306163449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f40c4320 msgr2=0x7fa4f41426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f40c4320 0x7fa4f41426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 -- 172.21.15.118:0/3306163449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40094f0 con 0x7fa4f4067d50 2024-08-18T13:51:58.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4ebfff700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f40c4320 0x7fa4f41426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:51:58.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f8ce3700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f4067d50 0x7fa4f4151f60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c4e0 tx=0x7fa4f0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4e8ff9700 1 -- 172.21.15.118:0/3306163449 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0011590 con 0x7fa4f4067d50 2024-08-18T13:51:58.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f4143df0 con 0x7fa4f4067d50 2024-08-18T13:51:58.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4e8ff9700 1 -- 172.21.15.118:0/3306163449 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f0011710 con 0x7fa4f4067d50 2024-08-18T13:51:58.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4e8ff9700 1 -- 172.21.15.118:0/3306163449 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0011ad0 con 0x7fa4f4067d50 2024-08-18T13:51:58.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.074+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f41513f0 con 0x7fa4f4067d50 2024-08-18T13:51:58.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.078+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f40713a0 con 0x7fa4f4067d50 2024-08-18T13:51:58.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.078+0000 7fa4e8ff9700 1 -- 172.21.15.118:0/3306163449 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa4f001c460 con 0x7fa4f4067d50 2024-08-18T13:51:58.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.078+0000 7fa4e8ff9700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa4e0060860 0x7fa4e0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.078+0000 7fa4e8ff9700 1 -- 172.21.15.118:0/3306163449 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fa4f007ce90 con 0x7fa4f4067d50 2024-08-18T13:51:58.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.078+0000 7fa4ebfff700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa4e0060860 0x7fa4e0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.078+0000 7fa4ebfff700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa4e0060860 0x7fa4e0062d10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4003080 tx=0x7fa4e4017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.090 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.086+0000 7fa4e8ff9700 1 -- 172.21.15.118:0/3306163449 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa4f004cc80 con 0x7fa4f4067d50 2024-08-18T13:51:58.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.246+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4f40c5b50 con 0x7fa4e0060860 2024-08-18T13:51:58.259 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.254+0000 7fa4e8ff9700 1 -- 172.21.15.118:0/3306163449 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+372617 (secure 0 0 0) 0x7fa4f40c5b50 con 0x7fa4e0060860 2024-08-18T13:51:58.265 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:58.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa4e0060860 msgr2=0x7fa4e0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa4e0060860 0x7fa4e0062d10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4003080 tx=0x7fa4e4017000 comp rx=0 tx=0).stop 2024-08-18T13:51:58.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f4067d50 msgr2=0x7fa4f4151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f4067d50 0x7fa4f4151f60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c4e0 tx=0x7fa4f0007ea0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.268 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 shutdown_connections 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fa4e0060860 0x7fa4e0062d10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40c46f0 0x7fa4f414b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f40c4320 0x7fa4f41426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 --2- 172.21.15.118:0/3306163449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa4f4067d50 0x7fa4f4151f60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 >> 172.21.15.118:0/3306163449 conn(0x7fa4f40c81e0 msgr2=0x7fa4f406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 shutdown_connections 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.262+0000 7fa4f9ce5700 1 -- 172.21.15.118:0/3306163449 wait complete. 2024-08-18T13:51:58.269 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:51:58.279 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-08-18T13:51:57.487883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1123528586,"num_objects":11047,"num_object_clones":0,"num_object_copies":43906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11047,"num_whiteouts":0,"num_read":22597,"num_read_kb":1390334,"num_write":28047,"num_write_kb":1250108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33741,"ondisk_log_size":33741,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4737384,"kb_used_data":2321128,"kb_used_omap":13,"kb_used_meta":2416178,"kb_avail":750237336,"statfs":{"total":773094113280,"available":768243032064,"internally_reserved":0,"allocated":2376835072,"data_stored":2374438700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2474166976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27132278,"num_objects":1405,"num_object_clones":0,"num_object_copies":5592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":9,"num_read_kb":916,"num_write":1574,"num_write_kb":27598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004102"},"pg_stats":[{"pgid":"2.1f","version":"53'151","reported_seq":214,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.743915+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:50:57.743915+0000","last_peered":"2024-08-18T13:50:57.743915+0000","last_clean":"2024-08-18T13:50:57.743915+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:50:57.743915+0000","last_undegraded":"2024-08-18T13:50:57.743915+0000","last_fullsized":"2024-08-18T13:50:57.743915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389178,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":28696,"num_write":150,"num_write_kb":8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766930+0000","last_change":"2024-08-18T13:43:55.555634+0000","last_active":"2024-08-18T13:51:51.766930+0000","last_peered":"2024-08-18T13:51:51.766930+0000","last_clean":"2024-08-18T13:51:51.766930+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:51:51.766930+0000","last_undegraded":"2024-08-18T13:51:51.766930+0000","last_fullsized":"2024-08-18T13:51:51.766930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064850389999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"54'676","reported_seq":3751,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.726737+0000","last_change":"2024-08-18T13:43:55.552107+0000","last_active":"2024-08-18T13:51:49.726737+0000","last_peered":"2024-08-18T13:51:49.726737+0000","last_clean":"2024-08-18T13:51:49.726737+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:51:49.726737+0000","last_undegraded":"2024-08-18T13:51:49.726737+0000","last_fullsized":"2024-08-18T13:51:49.726737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067303959999999996,"stat_sum":{"num_bytes":37566158,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":1013,"num_read_kb":36248,"num_write":673,"num_write_kb":41139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"54'78","reported_seq":126,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.670265+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:50:55.670265+0000","last_peered":"2024-08-18T13:50:55.670265+0000","last_clean":"2024-08-18T13:50:55.670265+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:50:55.670265+0000","last_undegraded":"2024-08-18T13:50:55.670265+0000","last_fullsized":"2024-08-18T13:50:55.670265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":75,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777966+0000","last_change":"2024-08-18T13:43:56.153208+0000","last_active":"2024-08-18T13:51:51.777966+0000","last_peered":"2024-08-18T13:51:51.777966+0000","last_clean":"2024-08-18T13:51:51.777966+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:51:51.777966+0000","last_undegraded":"2024-08-18T13:51:51.777966+0000","last_fullsized":"2024-08-18T13:51:51.777966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075430439999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"54'705","reported_seq":3559,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.547567+0000","last_change":"2024-08-18T13:43:55.555148+0000","last_active":"2024-08-18T13:51:49.547567+0000","last_peered":"2024-08-18T13:51:49.547567+0000","last_clean":"2024-08-18T13:51:49.547567+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:51:49.547567+0000","last_undegraded":"2024-08-18T13:51:49.547567+0000","last_fullsized":"2024-08-18T13:51:49.547567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075371290000000001,"stat_sum":{"num_bytes":29851183,"num_objects":370,"num_object_clones":0,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":792,"num_read_kb":20881,"num_write":708,"num_write_kb":37566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"54'194","reported_seq":259,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.297066+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:51:47.297066+0000","last_peered":"2024-08-18T13:51:47.297066+0000","last_clean":"2024-08-18T13:51:47.297066+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:51:47.297066+0000","last_undegraded":"2024-08-18T13:51:47.297066+0000","last_fullsized":"2024-08-18T13:51:47.297066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":69650,"num_write":192,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775007+0000","last_change":"2024-08-18T13:43:55.857350+0000","last_active":"2024-08-18T13:51:51.775007+0000","last_peered":"2024-08-18T13:51:51.775007+0000","last_clean":"2024-08-18T13:51:51.775007+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:51:51.775007+0000","last_undegraded":"2024-08-18T13:51:51.775007+0000","last_fullsized":"2024-08-18T13:51:51.775007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046085359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"54'680","reported_seq":3480,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.425160+0000","last_change":"2024-08-18T13:43:55.557895+0000","last_active":"2024-08-18T13:51:49.425160+0000","last_peered":"2024-08-18T13:51:49.425160+0000","last_clean":"2024-08-18T13:51:49.425160+0000","last_became_active":"2024-08-18T13:40:16.608743+0000","last_became_peered":"2024-08-18T13:40:16.608743+0000","last_unstale":"2024-08-18T13:51:49.425160+0000","last_undegraded":"2024-08-18T13:51:49.425160+0000","last_fullsized":"2024-08-18T13:51:49.425160+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:59:49.527489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080275980000000004,"stat_sum":{"num_bytes":28774140,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":727,"num_read_kb":13856,"num_write":680,"num_write_kb":28426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"53'23","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.459383+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:50:56.459383+0000","last_peered":"2024-08-18T13:50:56.459383+0000","last_clean":"2024-08-18T13:50:56.459383+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:50:56.459383+0000","last_undegraded":"2024-08-18T13:50:56.459383+0000","last_fullsized":"2024-08-18T13:50:56.459383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":25,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777014+0000","last_change":"2024-08-18T13:43:55.862914+0000","last_active":"2024-08-18T13:51:51.777014+0000","last_peered":"2024-08-18T13:51:51.777014+0000","last_clean":"2024-08-18T13:51:51.777014+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:51:51.777014+0000","last_undegraded":"2024-08-18T13:51:51.777014+0000","last_fullsized":"2024-08-18T13:51:51.777014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065299970000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"54'674","reported_seq":2902,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.448739+0000","last_change":"2024-08-18T13:43:55.555499+0000","last_active":"2024-08-18T13:51:49.448739+0000","last_peered":"2024-08-18T13:51:49.448739+0000","last_clean":"2024-08-18T13:51:49.448739+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:51:49.448739+0000","last_undegraded":"2024-08-18T13:51:49.448739+0000","last_fullsized":"2024-08-18T13:51:49.448739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078514629999999995,"stat_sum":{"num_bytes":11851705,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":166,"num_read_kb":1355,"num_write":673,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755374+0000","last_change":"2024-08-18T13:43:55.954106+0000","last_active":"2024-08-18T13:51:51.755374+0000","last_peered":"2024-08-18T13:51:51.755374+0000","last_clean":"2024-08-18T13:51:51.755374+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:51:51.755374+0000","last_undegraded":"2024-08-18T13:51:51.755374+0000","last_fullsized":"2024-08-18T13:51:51.755374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033699279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"54'215","reported_seq":274,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:53.770156+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:51:53.770156+0000","last_peered":"2024-08-18T13:51:53.770156+0000","last_clean":"2024-08-18T13:51:53.770156+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:51:53.770156+0000","last_undegraded":"2024-08-18T13:51:53.770156+0000","last_fullsized":"2024-08-18T13:51:53.770156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11951501,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":23,"num_read_kb":45068,"num_write":210,"num_write_kb":11823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"54'638","reported_seq":3281,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.214771+0000","last_change":"2024-08-18T13:43:55.558893+0000","last_active":"2024-08-18T13:51:49.214771+0000","last_peered":"2024-08-18T13:51:49.214771+0000","last_clean":"2024-08-18T13:51:49.214771+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:51:49.214771+0000","last_undegraded":"2024-08-18T13:51:49.214771+0000","last_fullsized":"2024-08-18T13:51:49.214771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059019540000000001,"stat_sum":{"num_bytes":16782769,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":581,"num_read_kb":14450,"num_write":639,"num_write_kb":24822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763887+0000","last_change":"2024-08-18T13:43:55.554204+0000","last_active":"2024-08-18T13:51:51.763887+0000","last_peered":"2024-08-18T13:51:51.763887+0000","last_clean":"2024-08-18T13:51:51.763887+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:51:51.763887+0000","last_undegraded":"2024-08-18T13:51:51.763887+0000","last_fullsized":"2024-08-18T13:51:51.763887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074252320000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"54'28","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.687997+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:50:56.687997+0000","last_peered":"2024-08-18T13:50:56.687997+0000","last_clean":"2024-08-18T13:50:56.687997+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:50:56.687997+0000","last_undegraded":"2024-08-18T13:50:56.687997+0000","last_fullsized":"2024-08-18T13:50:56.687997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":28,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"54'709","reported_seq":3799,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.722324+0000","last_change":"2024-08-18T13:43:55.558557+0000","last_active":"2024-08-18T13:51:49.722324+0000","last_peered":"2024-08-18T13:51:49.722324+0000","last_clean":"2024-08-18T13:51:49.722324+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:51:49.722324+0000","last_undegraded":"2024-08-18T13:51:49.722324+0000","last_fullsized":"2024-08-18T13:51:49.722324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057029610000000003,"stat_sum":{"num_bytes":33413248,"num_objects":347,"num_object_clones":0,"num_object_copies":1388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1028,"num_read_kb":27410,"num_write":707,"num_write_kb":41185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761963+0000","last_change":"2024-08-18T13:43:55.553624+0000","last_active":"2024-08-18T13:51:51.761963+0000","last_peered":"2024-08-18T13:51:51.761963+0000","last_clean":"2024-08-18T13:51:51.761963+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:51:51.761963+0000","last_undegraded":"2024-08-18T13:51:51.761963+0000","last_fullsized":"2024-08-18T13:51:51.761963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069243719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"53'13","reported_seq":57,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:50.648992+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:50:50.648992+0000","last_peered":"2024-08-18T13:50:50.648992+0000","last_clean":"2024-08-18T13:50:50.648992+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:50:50.648992+0000","last_undegraded":"2024-08-18T13:50:50.648992+0000","last_fullsized":"2024-08-18T13:50:50.648992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":11,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'683","reported_seq":3570,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.650342+0000","last_change":"2024-08-18T13:43:56.142589+0000","last_active":"2024-08-18T13:51:49.650342+0000","last_peered":"2024-08-18T13:51:49.650342+0000","last_clean":"2024-08-18T13:51:49.650342+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:51:49.650342+0000","last_undegraded":"2024-08-18T13:51:49.650342+0000","last_fullsized":"2024-08-18T13:51:49.650342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079287170000000001,"stat_sum":{"num_bytes":20263748,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":825,"num_read_kb":25932,"num_write":679,"num_write_kb":28331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:50.650367+0000","last_change":"2024-08-18T13:43:55.558187+0000","last_active":"2024-08-18T13:50:50.650367+0000","last_peered":"2024-08-18T13:50:50.650367+0000","last_clean":"2024-08-18T13:50:50.650367+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:50:50.650367+0000","last_undegraded":"2024-08-18T13:50:50.650367+0000","last_fullsized":"2024-08-18T13:50:50.650367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068791520000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"54'106","reported_seq":153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.256673+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:51:41.256673+0000","last_peered":"2024-08-18T13:51:41.256673+0000","last_clean":"2024-08-18T13:51:41.256673+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:51:41.256673+0000","last_undegraded":"2024-08-18T13:51:41.256673+0000","last_fullsized":"2024-08-18T13:51:41.256673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":108,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"54'617","reported_seq":3452,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.520510+0000","last_change":"2024-08-18T13:43:55.559559+0000","last_active":"2024-08-18T13:51:49.520510+0000","last_peered":"2024-08-18T13:51:49.520510+0000","last_clean":"2024-08-18T13:51:49.520510+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:51:49.520510+0000","last_undegraded":"2024-08-18T13:51:49.520510+0000","last_fullsized":"2024-08-18T13:51:49.520510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004078028,"stat_sum":{"num_bytes":24272349,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":773,"num_read_kb":25488,"num_write":617,"num_write_kb":28093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'93","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762461+0000","last_change":"2024-08-18T13:43:56.154543+0000","last_active":"2024-08-18T13:51:51.762461+0000","last_peered":"2024-08-18T13:51:51.762461+0000","last_clean":"2024-08-18T13:51:51.762461+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:51:51.762461+0000","last_undegraded":"2024-08-18T13:51:51.762461+0000","last_fullsized":"2024-08-18T13:51:51.762461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060661889999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"54'111","reported_seq":160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.305665+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:51:17.305665+0000","last_peered":"2024-08-18T13:51:17.305665+0000","last_clean":"2024-08-18T13:51:17.305665+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:51:17.305665+0000","last_undegraded":"2024-08-18T13:51:17.305665+0000","last_fullsized":"2024-08-18T13:51:17.305665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":111,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"54'720","reported_seq":2969,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.297825+0000","last_change":"2024-08-18T13:43:55.557702+0000","last_active":"2024-08-18T13:51:49.297825+0000","last_peered":"2024-08-18T13:51:49.297825+0000","last_clean":"2024-08-18T13:51:49.297825+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:51:49.297825+0000","last_undegraded":"2024-08-18T13:51:49.297825+0000","last_fullsized":"2024-08-18T13:51:49.297825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079796720000000002,"stat_sum":{"num_bytes":17871290,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":187,"num_read_kb":7768,"num_write":717,"num_write_kb":21887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781486+0000","last_change":"2024-08-18T13:43:55.555491+0000","last_active":"2024-08-18T13:51:51.781486+0000","last_peered":"2024-08-18T13:51:51.781486+0000","last_clean":"2024-08-18T13:51:51.781486+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:51:51.781486+0000","last_undegraded":"2024-08-18T13:51:51.781486+0000","last_fullsized":"2024-08-18T13:51:51.781486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054340660000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"54'66","reported_seq":116,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:48.746876+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:51:48.746876+0000","last_peered":"2024-08-18T13:51:48.746876+0000","last_clean":"2024-08-18T13:51:48.746876+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:51:48.746876+0000","last_undegraded":"2024-08-18T13:51:48.746876+0000","last_fullsized":"2024-08-18T13:51:48.746876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":184,"num_write":164,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"54'714","reported_seq":2929,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.710846+0000","last_change":"2024-08-18T13:43:55.550748+0000","last_active":"2024-08-18T13:51:49.710846+0000","last_peered":"2024-08-18T13:51:49.710846+0000","last_clean":"2024-08-18T13:51:49.710846+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:51:49.710846+0000","last_undegraded":"2024-08-18T13:51:49.710846+0000","last_fullsized":"2024-08-18T13:51:49.710846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057306029999999999,"stat_sum":{"num_bytes":13242151,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":153,"num_read_kb":5341,"num_write":712,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758218+0000","last_change":"2024-08-18T13:43:56.143836+0000","last_active":"2024-08-18T13:51:51.758218+0000","last_peered":"2024-08-18T13:51:51.758218+0000","last_clean":"2024-08-18T13:51:51.758218+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:51:51.758218+0000","last_undegraded":"2024-08-18T13:51:51.758218+0000","last_fullsized":"2024-08-18T13:51:51.758218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064599770000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"54'289","reported_seq":337,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:33.863944+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:51:33.863944+0000","last_peered":"2024-08-18T13:51:33.863944+0000","last_clean":"2024-08-18T13:51:33.863944+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:51:33.863944+0000","last_undegraded":"2024-08-18T13:51:33.863944+0000","last_fullsized":"2024-08-18T13:51:33.863944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":12176,"num_write":284,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"54'704","reported_seq":3425,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.623916+0000","last_change":"2024-08-18T13:43:56.151855+0000","last_active":"2024-08-18T13:51:49.623916+0000","last_peered":"2024-08-18T13:51:49.623916+0000","last_clean":"2024-08-18T13:51:49.623916+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:51:49.623916+0000","last_undegraded":"2024-08-18T13:51:49.623916+0000","last_fullsized":"2024-08-18T13:51:49.623916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060004689999999996,"stat_sum":{"num_bytes":28446730,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":659,"num_read_kb":21792,"num_write":702,"num_write_kb":28155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"54'84","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775443+0000","last_change":"2024-08-18T13:43:55.863531+0000","last_active":"2024-08-18T13:51:51.775443+0000","last_peered":"2024-08-18T13:51:51.775443+0000","last_clean":"2024-08-18T13:51:51.775443+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:51:51.775443+0000","last_undegraded":"2024-08-18T13:51:51.775443+0000","last_fullsized":"2024-08-18T13:51:51.775443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006455484,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"54'657","reported_seq":700,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.994466+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:51:38.994466+0000","last_peered":"2024-08-18T13:51:38.994466+0000","last_clean":"2024-08-18T13:51:38.994466+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:51:38.994466+0000","last_undegraded":"2024-08-18T13:51:38.994466+0000","last_fullsized":"2024-08-18T13:51:38.994466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"54'691","reported_seq":3343,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.557816+0000","last_change":"2024-08-18T13:43:56.141004+0000","last_active":"2024-08-18T13:51:49.557816+0000","last_peered":"2024-08-18T13:51:49.557816+0000","last_clean":"2024-08-18T13:51:49.557816+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:51:49.557816+0000","last_undegraded":"2024-08-18T13:51:49.557816+0000","last_fullsized":"2024-08-18T13:51:49.557816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067583870000000002,"stat_sum":{"num_bytes":38683931,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":590,"num_read_kb":16518,"num_write":694,"num_write_kb":42167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"54'97","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758746+0000","last_change":"2024-08-18T13:43:55.559256+0000","last_active":"2024-08-18T13:51:51.758746+0000","last_peered":"2024-08-18T13:51:51.758746+0000","last_clean":"2024-08-18T13:51:51.758746+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:51:51.758746+0000","last_undegraded":"2024-08-18T13:51:51.758746+0000","last_fullsized":"2024-08-18T13:51:51.758746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005854756,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"53'383","reported_seq":438,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.690458+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:50:55.690458+0000","last_peered":"2024-08-18T13:50:55.690458+0000","last_clean":"2024-08-18T13:50:55.690458+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:50:55.690458+0000","last_undegraded":"2024-08-18T13:50:55.690458+0000","last_fullsized":"2024-08-18T13:50:55.690458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":43191,"num_write":382,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"54'661","reported_seq":4460,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.576473+0000","last_change":"2024-08-18T13:43:56.144223+0000","last_active":"2024-08-18T13:51:49.576473+0000","last_peered":"2024-08-18T13:51:49.576473+0000","last_clean":"2024-08-18T13:51:49.576473+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:51:49.576473+0000","last_undegraded":"2024-08-18T13:51:49.576473+0000","last_fullsized":"2024-08-18T13:51:49.576473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067969390000000001,"stat_sum":{"num_bytes":51061167,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":1737,"num_read_kb":43040,"num_write":661,"num_write_kb":53481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"54'83","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758102+0000","last_change":"2024-08-18T13:43:56.143663+0000","last_active":"2024-08-18T13:51:51.758102+0000","last_peered":"2024-08-18T13:51:51.758102+0000","last_clean":"2024-08-18T13:51:51.758102+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:51:51.758102+0000","last_undegraded":"2024-08-18T13:51:51.758102+0000","last_fullsized":"2024-08-18T13:51:51.758102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063915389999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"53'120","reported_seq":283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.353455+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:50:51.353455+0000","last_peered":"2024-08-18T13:50:51.353455+0000","last_clean":"2024-08-18T13:50:51.353455+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:50:51.353455+0000","last_undegraded":"2024-08-18T13:50:51.353455+0000","last_fullsized":"2024-08-18T13:50:51.353455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":638,"num_write":117,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"54'643","reported_seq":3597,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.312624+0000","last_change":"2024-08-18T13:43:56.143097+0000","last_active":"2024-08-18T13:51:49.312624+0000","last_peered":"2024-08-18T13:51:49.312624+0000","last_clean":"2024-08-18T13:51:49.312624+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:51:49.312624+0000","last_undegraded":"2024-08-18T13:51:49.312624+0000","last_fullsized":"2024-08-18T13:51:49.312624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079242040000000007,"stat_sum":{"num_bytes":41790406,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":892,"num_read_kb":23319,"num_write":639,"num_write_kb":45248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'82","reported_seq":2146,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762481+0000","last_change":"2024-08-18T13:43:56.149372+0000","last_active":"2024-08-18T13:51:51.762481+0000","last_peered":"2024-08-18T13:51:51.762481+0000","last_clean":"2024-08-18T13:51:51.762481+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:51:51.762481+0000","last_undegraded":"2024-08-18T13:51:51.762481+0000","last_fullsized":"2024-08-18T13:51:51.762481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041495459999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"54'234","reported_seq":290,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.626190+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:50:57.626190+0000","last_peered":"2024-08-18T13:50:57.626190+0000","last_clean":"2024-08-18T13:50:57.626190+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:50:57.626190+0000","last_undegraded":"2024-08-18T13:50:57.626190+0000","last_fullsized":"2024-08-18T13:50:57.626190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":49152,"num_write":231,"num_write_kb":12421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"54'716","reported_seq":3216,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.491284+0000","last_change":"2024-08-18T13:43:56.153929+0000","last_active":"2024-08-18T13:51:49.491284+0000","last_peered":"2024-08-18T13:51:49.491284+0000","last_clean":"2024-08-18T13:51:49.491284+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:51:49.491284+0000","last_undegraded":"2024-08-18T13:51:49.491284+0000","last_fullsized":"2024-08-18T13:51:49.491284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058974379999999996,"stat_sum":{"num_bytes":30462084,"num_objects":352,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":439,"num_read_kb":15189,"num_write":710,"num_write_kb":30067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779966+0000","last_change":"2024-08-18T13:43:55.552534+0000","last_active":"2024-08-18T13:51:51.779966+0000","last_peered":"2024-08-18T13:51:51.779966+0000","last_clean":"2024-08-18T13:51:51.779966+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:51:51.779966+0000","last_undegraded":"2024-08-18T13:51:51.779966+0000","last_fullsized":"2024-08-18T13:51:51.779966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068182370000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"54'982","reported_seq":1027,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.365615+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:51:17.365615+0000","last_peered":"2024-08-18T13:51:17.365615+0000","last_clean":"2024-08-18T13:51:17.365615+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:51:17.365615+0000","last_undegraded":"2024-08-18T13:51:17.365615+0000","last_fullsized":"2024-08-18T13:51:17.365615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2087,"num_write":979,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:49:30.943967+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:49:30.943967+0000","last_peered":"2024-08-18T13:49:30.943967+0000","last_clean":"2024-08-18T13:49:30.943967+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:49:30.943967+0000","last_undegraded":"2024-08-18T13:49:30.943967+0000","last_fullsized":"2024-08-18T13:49:30.943967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"54'664","reported_seq":3524,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.497597+0000","last_change":"2024-08-18T13:43:55.956673+0000","last_active":"2024-08-18T13:51:49.497597+0000","last_peered":"2024-08-18T13:51:49.497597+0000","last_clean":"2024-08-18T13:51:49.497597+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:51:49.497597+0000","last_undegraded":"2024-08-18T13:51:49.497597+0000","last_fullsized":"2024-08-18T13:51:49.497597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035684480000000001,"stat_sum":{"num_bytes":39930170,"num_objects":364,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":798,"num_read_kb":26403,"num_write":662,"num_write_kb":47533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779241+0000","last_change":"2024-08-18T13:43:55.552313+0000","last_active":"2024-08-18T13:51:51.779241+0000","last_peered":"2024-08-18T13:51:51.779241+0000","last_clean":"2024-08-18T13:51:51.779241+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:51:51.779241+0000","last_undegraded":"2024-08-18T13:51:51.779241+0000","last_fullsized":"2024-08-18T13:51:51.779241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057711639999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"54'76","reported_seq":118,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.291821+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:51:47.291821+0000","last_peered":"2024-08-18T13:51:47.291821+0000","last_clean":"2024-08-18T13:51:47.291821+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:51:47.291821+0000","last_undegraded":"2024-08-18T13:51:47.291821+0000","last_fullsized":"2024-08-18T13:51:47.291821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"54'688","reported_seq":4127,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.578882+0000","last_change":"2024-08-18T13:43:55.953251+0000","last_active":"2024-08-18T13:51:49.578882+0000","last_peered":"2024-08-18T13:51:49.578882+0000","last_clean":"2024-08-18T13:51:49.578882+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:51:49.578882+0000","last_undegraded":"2024-08-18T13:51:49.578882+0000","last_fullsized":"2024-08-18T13:51:49.578882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032433420000000002,"stat_sum":{"num_bytes":54557211,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1377,"num_read_kb":50055,"num_write":687,"num_write_kb":57734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"54'95","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.756325+0000","last_change":"2024-08-18T13:43:55.918416+0000","last_active":"2024-08-18T13:51:51.756325+0000","last_peered":"2024-08-18T13:51:51.756325+0000","last_clean":"2024-08-18T13:51:51.756325+0000","last_became_active":"2024-08-18T13:40:16.614814+0000","last_became_peered":"2024-08-18T13:40:16.614814+0000","last_unstale":"2024-08-18T13:51:51.756325+0000","last_undegraded":"2024-08-18T13:51:51.756325+0000","last_fullsized":"2024-08-18T13:51:51.756325+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:02:39.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033975450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'31","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.186584+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:50:56.186584+0000","last_peered":"2024-08-18T13:50:56.186584+0000","last_clean":"2024-08-18T13:50:56.186584+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:50:56.186584+0000","last_undegraded":"2024-08-18T13:50:56.186584+0000","last_fullsized":"2024-08-18T13:50:56.186584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":33,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"54'670","reported_seq":3234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.448131+0000","last_change":"2024-08-18T13:43:56.153281+0000","last_active":"2024-08-18T13:51:49.448131+0000","last_peered":"2024-08-18T13:51:49.448131+0000","last_clean":"2024-08-18T13:51:49.448131+0000","last_became_active":"2024-08-18T13:40:16.609908+0000","last_became_peered":"2024-08-18T13:40:16.609908+0000","last_unstale":"2024-08-18T13:51:49.448131+0000","last_undegraded":"2024-08-18T13:51:49.448131+0000","last_fullsized":"2024-08-18T13:51:49.448131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:35:52.383892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069871079999999997,"stat_sum":{"num_bytes":26719276,"num_objects":358,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":491,"num_read_kb":18148,"num_write":671,"num_write_kb":26414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4,5],"acting":[0,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784591+0000","last_change":"2024-08-18T13:43:55.553120+0000","last_active":"2024-08-18T13:51:51.784591+0000","last_peered":"2024-08-18T13:51:51.784591+0000","last_clean":"2024-08-18T13:51:51.784591+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:51:51.784591+0000","last_undegraded":"2024-08-18T13:51:51.784591+0000","last_fullsized":"2024-08-18T13:51:51.784591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057951929999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"53'29","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.394499+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:50:56.394499+0000","last_peered":"2024-08-18T13:50:56.394499+0000","last_clean":"2024-08-18T13:50:56.394499+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:50:56.394499+0000","last_undegraded":"2024-08-18T13:50:56.394499+0000","last_fullsized":"2024-08-18T13:50:56.394499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":30,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"54'688","reported_seq":2818,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.687818+0000","last_change":"2024-08-18T13:43:55.859666+0000","last_active":"2024-08-18T13:51:49.687818+0000","last_peered":"2024-08-18T13:51:49.687818+0000","last_clean":"2024-08-18T13:51:49.687818+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:51:49.687818+0000","last_undegraded":"2024-08-18T13:51:49.687818+0000","last_fullsized":"2024-08-18T13:51:49.687818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070733389999999997,"stat_sum":{"num_bytes":17282460,"num_objects":346,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":68,"num_read_kb":8224,"num_write":686,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"54'66","reported_seq":2129,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783626+0000","last_change":"2024-08-18T13:43:55.551809+0000","last_active":"2024-08-18T13:51:51.783626+0000","last_peered":"2024-08-18T13:51:51.783626+0000","last_clean":"2024-08-18T13:51:51.783626+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:51:51.783626+0000","last_undegraded":"2024-08-18T13:51:51.783626+0000","last_fullsized":"2024-08-18T13:51:51.783626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060906930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"53'26","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.647720+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:50:55.647720+0000","last_peered":"2024-08-18T13:50:55.647720+0000","last_clean":"2024-08-18T13:50:55.647720+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:50:55.647720+0000","last_undegraded":"2024-08-18T13:50:55.647720+0000","last_fullsized":"2024-08-18T13:50:55.647720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":27,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"54'695","reported_seq":3630,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.666776+0000","last_change":"2024-08-18T13:43:55.557389+0000","last_active":"2024-08-18T13:51:49.666776+0000","last_peered":"2024-08-18T13:51:49.666776+0000","last_clean":"2024-08-18T13:51:49.666776+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:51:49.666776+0000","last_undegraded":"2024-08-18T13:51:49.666776+0000","last_fullsized":"2024-08-18T13:51:49.666776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011606655,"stat_sum":{"num_bytes":44119789,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":873,"num_read_kb":37157,"num_write":696,"num_write_kb":47501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"54'91","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784041+0000","last_change":"2024-08-18T13:43:55.553915+0000","last_active":"2024-08-18T13:51:51.784041+0000","last_peered":"2024-08-18T13:51:51.784041+0000","last_clean":"2024-08-18T13:51:51.784041+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:51:51.784041+0000","last_undegraded":"2024-08-18T13:51:51.784041+0000","last_fullsized":"2024-08-18T13:51:51.784041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053019490000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"53'1204","reported_seq":1276,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.252643+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:51:17.252643+0000","last_peered":"2024-08-18T13:51:17.252643+0000","last_clean":"2024-08-18T13:51:17.252643+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:51:17.252643+0000","last_undegraded":"2024-08-18T13:51:17.252643+0000","last_fullsized":"2024-08-18T13:51:17.252643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":78,"num_read_kb":12399,"num_write":1202,"num_write_kb":8830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'657","reported_seq":3516,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.665369+0000","last_change":"2024-08-18T13:43:56.154837+0000","last_active":"2024-08-18T13:51:49.665369+0000","last_peered":"2024-08-18T13:51:49.665369+0000","last_clean":"2024-08-18T13:51:49.665369+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:51:49.665369+0000","last_undegraded":"2024-08-18T13:51:49.665369+0000","last_fullsized":"2024-08-18T13:51:49.665369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054104280000000001,"stat_sum":{"num_bytes":33280018,"num_objects":296,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296,"num_whiteouts":0,"num_read":797,"num_read_kb":22024,"num_write":655,"num_write_kb":32783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757105+0000","last_change":"2024-08-18T13:43:55.959459+0000","last_active":"2024-08-18T13:51:51.757105+0000","last_peered":"2024-08-18T13:51:51.757105+0000","last_clean":"2024-08-18T13:51:51.757105+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:51:51.757105+0000","last_undegraded":"2024-08-18T13:51:51.757105+0000","last_fullsized":"2024-08-18T13:51:51.757105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044255090000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"54'5","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.846481+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:50:51.846481+0000","last_peered":"2024-08-18T13:50:51.846481+0000","last_clean":"2024-08-18T13:50:51.846481+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:50:51.846481+0000","last_undegraded":"2024-08-18T13:50:51.846481+0000","last_fullsized":"2024-08-18T13:50:51.846481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"54'660","reported_seq":2779,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.587481+0000","last_change":"2024-08-18T13:43:55.555242+0000","last_active":"2024-08-18T13:51:49.587481+0000","last_peered":"2024-08-18T13:51:49.587481+0000","last_clean":"2024-08-18T13:51:49.587481+0000","last_became_active":"2024-08-18T13:40:16.915369+0000","last_became_peered":"2024-08-18T13:40:16.915369+0000","last_unstale":"2024-08-18T13:51:49.587481+0000","last_undegraded":"2024-08-18T13:51:49.587481+0000","last_fullsized":"2024-08-18T13:51:49.587481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:50:23.633304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005619059,"stat_sum":{"num_bytes":16977484,"num_objects":353,"num_object_clones":0,"num_object_copies":1412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":46,"num_read_kb":4277,"num_write":660,"num_write_kb":20975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,6],"acting":[3,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"54'101","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784667+0000","last_change":"2024-08-18T13:43:55.551514+0000","last_active":"2024-08-18T13:51:51.784667+0000","last_peered":"2024-08-18T13:51:51.784667+0000","last_clean":"2024-08-18T13:51:51.784667+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:51:51.784667+0000","last_undegraded":"2024-08-18T13:51:51.784667+0000","last_fullsized":"2024-08-18T13:51:51.784667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064019180000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"53'2","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:50:28.547615+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:50:28.547615+0000","last_peered":"2024-08-18T13:50:28.547615+0000","last_clean":"2024-08-18T13:50:28.547615+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:50:28.547615+0000","last_undegraded":"2024-08-18T13:50:28.547615+0000","last_fullsized":"2024-08-18T13:50:28.547615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"54'625","reported_seq":3598,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.686816+0000","last_change":"2024-08-18T13:43:55.554811+0000","last_active":"2024-08-18T13:51:49.686816+0000","last_peered":"2024-08-18T13:51:49.686816+0000","last_clean":"2024-08-18T13:51:49.686816+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:51:49.686816+0000","last_undegraded":"2024-08-18T13:51:49.686816+0000","last_fullsized":"2024-08-18T13:51:49.686816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074125900000000002,"stat_sum":{"num_bytes":40693231,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":911,"num_read_kb":37155,"num_write":622,"num_write_kb":48255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778223+0000","last_change":"2024-08-18T13:43:56.154386+0000","last_active":"2024-08-18T13:51:51.778223+0000","last_peered":"2024-08-18T13:51:51.778223+0000","last_clean":"2024-08-18T13:51:51.778223+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:51:51.778223+0000","last_undegraded":"2024-08-18T13:51:51.778223+0000","last_fullsized":"2024-08-18T13:51:51.778223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059765870000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"54'216","reported_seq":283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.281097+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:51:41.281097+0000","last_peered":"2024-08-18T13:51:41.281097+0000","last_clean":"2024-08-18T13:51:41.281097+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:51:41.281097+0000","last_undegraded":"2024-08-18T13:51:41.281097+0000","last_fullsized":"2024-08-18T13:51:41.281097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":45098,"num_write":215,"num_write_kb":8331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'629","reported_seq":3941,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.453295+0000","last_change":"2024-08-18T13:43:55.862627+0000","last_active":"2024-08-18T13:51:49.453295+0000","last_peered":"2024-08-18T13:51:49.453295+0000","last_clean":"2024-08-18T13:51:49.453295+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:51:49.453295+0000","last_undegraded":"2024-08-18T13:51:49.453295+0000","last_fullsized":"2024-08-18T13:51:49.453295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084356780000000003,"stat_sum":{"num_bytes":50488256,"num_objects":306,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":1250,"num_read_kb":43248,"num_write":629,"num_write_kb":49603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'84","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779153+0000","last_change":"2024-08-18T13:43:55.551442+0000","last_active":"2024-08-18T13:51:51.779153+0000","last_peered":"2024-08-18T13:51:51.779153+0000","last_clean":"2024-08-18T13:51:51.779153+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:51:51.779153+0000","last_undegraded":"2024-08-18T13:51:51.779153+0000","last_fullsized":"2024-08-18T13:51:51.779153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065977270000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"54'232","reported_seq":305,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.581418+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:51:41.581418+0000","last_peered":"2024-08-18T13:51:41.581418+0000","last_clean":"2024-08-18T13:51:41.581418+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:51:41.581418+0000","last_undegraded":"2024-08-18T13:51:41.581418+0000","last_fullsized":"2024-08-18T13:51:41.581418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583572,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":53278,"num_write":225,"num_write_kb":12431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"54'671","reported_seq":3262,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.458342+0000","last_change":"2024-08-18T13:43:56.142175+0000","last_active":"2024-08-18T13:51:49.458342+0000","last_peered":"2024-08-18T13:51:49.458342+0000","last_clean":"2024-08-18T13:51:49.458342+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:51:49.458342+0000","last_undegraded":"2024-08-18T13:51:49.458342+0000","last_fullsized":"2024-08-18T13:51:49.458342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076714670000000004,"stat_sum":{"num_bytes":16086174,"num_objects":343,"num_object_clones":0,"num_object_copies":1372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":529,"num_read_kb":12373,"num_write":668,"num_write_kb":24206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"54'86","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783436+0000","last_change":"2024-08-18T13:43:55.559262+0000","last_active":"2024-08-18T13:51:51.783436+0000","last_peered":"2024-08-18T13:51:51.783436+0000","last_clean":"2024-08-18T13:51:51.783436+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:51:51.783436+0000","last_undegraded":"2024-08-18T13:51:51.783436+0000","last_fullsized":"2024-08-18T13:51:51.783436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039212459999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"53'169","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.276349+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:50:57.276349+0000","last_peered":"2024-08-18T13:50:57.276349+0000","last_clean":"2024-08-18T13:50:57.276349+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:50:57.276349+0000","last_undegraded":"2024-08-18T13:50:57.276349+0000","last_fullsized":"2024-08-18T13:50:57.276349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8216,"num_write":168,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'644","reported_seq":3278,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.432661+0000","last_change":"2024-08-18T13:43:55.559022+0000","last_active":"2024-08-18T13:51:49.432661+0000","last_peered":"2024-08-18T13:51:49.432661+0000","last_clean":"2024-08-18T13:51:49.432661+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:51:49.432661+0000","last_undegraded":"2024-08-18T13:51:49.432661+0000","last_fullsized":"2024-08-18T13:51:49.432661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059375670000000004,"stat_sum":{"num_bytes":15740586,"num_objects":325,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":572,"num_read_kb":18359,"num_write":643,"num_write_kb":23895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"54'89","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778006+0000","last_change":"2024-08-18T13:43:56.153537+0000","last_active":"2024-08-18T13:51:51.778006+0000","last_peered":"2024-08-18T13:51:51.778006+0000","last_clean":"2024-08-18T13:51:51.778006+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:51:51.778006+0000","last_undegraded":"2024-08-18T13:51:51.778006+0000","last_fullsized":"2024-08-18T13:51:51.778006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057226309999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"54'233","reported_seq":282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.891732+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:50:56.891732+0000","last_peered":"2024-08-18T13:50:56.891732+0000","last_clean":"2024-08-18T13:50:56.891732+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:50:56.891732+0000","last_undegraded":"2024-08-18T13:50:56.891732+0000","last_fullsized":"2024-08-18T13:50:56.891732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":20397,"num_write":232,"num_write_kb":4254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"54'677","reported_seq":3169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.442883+0000","last_change":"2024-08-18T13:43:55.553346+0000","last_active":"2024-08-18T13:51:49.442883+0000","last_peered":"2024-08-18T13:51:49.442883+0000","last_clean":"2024-08-18T13:51:49.442883+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:51:49.442883+0000","last_undegraded":"2024-08-18T13:51:49.442883+0000","last_fullsized":"2024-08-18T13:51:49.442883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073620359999999998,"stat_sum":{"num_bytes":31240743,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":431,"num_read_kb":13577,"num_write":670,"num_write_kb":34133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"54'106","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778915+0000","last_change":"2024-08-18T13:43:55.861465+0000","last_active":"2024-08-18T13:51:51.778915+0000","last_peered":"2024-08-18T13:51:51.778915+0000","last_clean":"2024-08-18T13:51:51.778915+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:51:51.778915+0000","last_undegraded":"2024-08-18T13:51:51.778915+0000","last_fullsized":"2024-08-18T13:51:51.778915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059613369999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"54'9","reported_seq":51,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.581557+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:51:41.581557+0000","last_peered":"2024-08-18T13:51:41.581557+0000","last_clean":"2024-08-18T13:51:41.581557+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:51:41.581557+0000","last_undegraded":"2024-08-18T13:51:41.581557+0000","last_fullsized":"2024-08-18T13:51:41.581557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"54'699","reported_seq":3480,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.480893+0000","last_change":"2024-08-18T13:43:55.553902+0000","last_active":"2024-08-18T13:51:49.480893+0000","last_peered":"2024-08-18T13:51:49.480893+0000","last_clean":"2024-08-18T13:51:49.480893+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:51:49.480893+0000","last_undegraded":"2024-08-18T13:51:49.480893+0000","last_fullsized":"2024-08-18T13:51:49.480893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076808309999999999,"stat_sum":{"num_bytes":29636158,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":719,"num_read_kb":24704,"num_write":698,"num_write_kb":38783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"54'650","reported_seq":3292,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.034549+0000","last_change":"2024-08-18T13:43:56.152924+0000","last_active":"2024-08-18T13:51:49.034549+0000","last_peered":"2024-08-18T13:51:49.034549+0000","last_clean":"2024-08-18T13:51:49.034549+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:51:49.034549+0000","last_undegraded":"2024-08-18T13:51:49.034549+0000","last_fullsized":"2024-08-18T13:51:49.034549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068864499999999997,"stat_sum":{"num_bytes":24960867,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":580,"num_read_kb":20952,"num_write":650,"num_write_kb":24715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"54'91","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761698+0000","last_change":"2024-08-18T13:43:56.139883+0000","last_active":"2024-08-18T13:51:51.761698+0000","last_peered":"2024-08-18T13:51:51.761698+0000","last_clean":"2024-08-18T13:51:51.761698+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:51:51.761698+0000","last_undegraded":"2024-08-18T13:51:51.761698+0000","last_fullsized":"2024-08-18T13:51:51.761698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066095759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"54'199","reported_seq":252,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:42.157051+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:51:42.157051+0000","last_peered":"2024-08-18T13:51:42.157051+0000","last_clean":"2024-08-18T13:51:42.157051+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:51:42.157051+0000","last_undegraded":"2024-08-18T13:51:42.157051+0000","last_fullsized":"2024-08-18T13:51:42.157051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":197,"num_write_kb":12441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"54'658","reported_seq":3187,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.395492+0000","last_change":"2024-08-18T13:43:55.958984+0000","last_active":"2024-08-18T13:51:49.395492+0000","last_peered":"2024-08-18T13:51:49.395492+0000","last_clean":"2024-08-18T13:51:49.395492+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:51:49.395492+0000","last_undegraded":"2024-08-18T13:51:49.395492+0000","last_fullsized":"2024-08-18T13:51:49.395492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047613839999999996,"stat_sum":{"num_bytes":20867535,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":467,"num_read_kb":16390,"num_write":656,"num_write_kb":20726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764935+0000","last_change":"2024-08-18T13:43:55.552907+0000","last_active":"2024-08-18T13:51:51.764935+0000","last_peered":"2024-08-18T13:51:51.764935+0000","last_clean":"2024-08-18T13:51:51.764935+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:51:51.764935+0000","last_undegraded":"2024-08-18T13:51:51.764935+0000","last_fullsized":"2024-08-18T13:51:51.764935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068339799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"53'103","reported_seq":150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.907236+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:50:51.907236+0000","last_peered":"2024-08-18T13:50:51.907236+0000","last_clean":"2024-08-18T13:50:51.907236+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:50:51.907236+0000","last_undegraded":"2024-08-18T13:50:51.907236+0000","last_fullsized":"2024-08-18T13:50:51.907236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":103,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"54'131","reported_seq":186,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:34.672086+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:51:34.672086+0000","last_peered":"2024-08-18T13:51:34.672086+0000","last_clean":"2024-08-18T13:51:34.672086+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:51:34.672086+0000","last_undegraded":"2024-08-18T13:51:34.672086+0000","last_fullsized":"2024-08-18T13:51:34.672086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":128,"num_write_kb":12417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"54'91","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.768070+0000","last_change":"2024-08-18T13:43:56.154716+0000","last_active":"2024-08-18T13:51:51.768070+0000","last_peered":"2024-08-18T13:51:51.768070+0000","last_clean":"2024-08-18T13:51:51.768070+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:51:51.768070+0000","last_undegraded":"2024-08-18T13:51:51.768070+0000","last_fullsized":"2024-08-18T13:51:51.768070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053680339999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"54'664","reported_seq":3634,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.568805+0000","last_change":"2024-08-18T13:43:55.917879+0000","last_active":"2024-08-18T13:51:49.568805+0000","last_peered":"2024-08-18T13:51:49.568805+0000","last_clean":"2024-08-18T13:51:49.568805+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:51:49.568805+0000","last_undegraded":"2024-08-18T13:51:49.568805+0000","last_fullsized":"2024-08-18T13:51:49.568805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035122529999999999,"stat_sum":{"num_bytes":30328916,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":908,"num_read_kb":29115,"num_write":663,"num_write_kb":34071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"54'14","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.993056+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:51:38.993056+0000","last_peered":"2024-08-18T13:51:38.993056+0000","last_clean":"2024-08-18T13:51:38.993056+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:51:38.993056+0000","last_undegraded":"2024-08-18T13:51:38.993056+0000","last_fullsized":"2024-08-18T13:51:38.993056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":19,"num_write_kb":4249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757533+0000","last_change":"2024-08-18T13:43:55.555770+0000","last_active":"2024-08-18T13:51:51.757533+0000","last_peered":"2024-08-18T13:51:51.757533+0000","last_clean":"2024-08-18T13:51:51.757533+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:51:51.757533+0000","last_undegraded":"2024-08-18T13:51:51.757533+0000","last_fullsized":"2024-08-18T13:51:51.757533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072935329999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"54'656","reported_seq":3234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.680798+0000","last_change":"2024-08-18T13:43:55.555171+0000","last_active":"2024-08-18T13:51:49.680798+0000","last_peered":"2024-08-18T13:51:49.680798+0000","last_clean":"2024-08-18T13:51:49.680798+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:51:49.680798+0000","last_undegraded":"2024-08-18T13:51:49.680798+0000","last_fullsized":"2024-08-18T13:51:49.680798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063112100000000003,"stat_sum":{"num_bytes":21201155,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":516,"num_read_kb":24278,"num_write":657,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"54'107","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764519+0000","last_change":"2024-08-18T13:43:55.959563+0000","last_active":"2024-08-18T13:51:51.764519+0000","last_peered":"2024-08-18T13:51:51.764519+0000","last_clean":"2024-08-18T13:51:51.764519+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:51:51.764519+0000","last_undegraded":"2024-08-18T13:51:51.764519+0000","last_fullsized":"2024-08-18T13:51:51.764519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045422170000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"54'110","reported_seq":2173,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776910+0000","last_change":"2024-08-18T13:43:55.919976+0000","last_active":"2024-08-18T13:51:51.776910+0000","last_peered":"2024-08-18T13:51:51.776910+0000","last_clean":"2024-08-18T13:51:51.776910+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:51:51.776910+0000","last_undegraded":"2024-08-18T13:51:51.776910+0000","last_fullsized":"2024-08-18T13:51:51.776910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032950929999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"54'101","reported_seq":2174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757889+0000","last_change":"2024-08-18T13:43:55.863304+0000","last_active":"2024-08-18T13:51:51.757889+0000","last_peered":"2024-08-18T13:51:51.757889+0000","last_clean":"2024-08-18T13:51:51.757889+0000","last_became_active":"2024-08-18T13:40:16.610568+0000","last_became_peered":"2024-08-18T13:40:16.610568+0000","last_unstale":"2024-08-18T13:51:51.757889+0000","last_undegraded":"2024-08-18T13:51:51.757889+0000","last_fullsized":"2024-08-18T13:51:51.757889+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:53:53.687401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066503589999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"54'99","reported_seq":2163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766416+0000","last_change":"2024-08-18T13:43:56.153594+0000","last_active":"2024-08-18T13:51:51.766416+0000","last_peered":"2024-08-18T13:51:51.766416+0000","last_clean":"2024-08-18T13:51:51.766416+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:51:51.766416+0000","last_undegraded":"2024-08-18T13:51:51.766416+0000","last_fullsized":"2024-08-18T13:51:51.766416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056450420000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"54'80","reported_seq":2143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764106+0000","last_change":"2024-08-18T13:43:55.554865+0000","last_active":"2024-08-18T13:51:51.764106+0000","last_peered":"2024-08-18T13:51:51.764106+0000","last_clean":"2024-08-18T13:51:51.764106+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:51:51.764106+0000","last_undegraded":"2024-08-18T13:51:51.764106+0000","last_fullsized":"2024-08-18T13:51:51.764106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068042459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"54'74","reported_seq":2138,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761397+0000","last_change":"2024-08-18T13:43:56.142479+0000","last_active":"2024-08-18T13:51:51.761397+0000","last_peered":"2024-08-18T13:51:51.761397+0000","last_clean":"2024-08-18T13:51:51.761397+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:51:51.761397+0000","last_undegraded":"2024-08-18T13:51:51.761397+0000","last_fullsized":"2024-08-18T13:51:51.761397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079157710000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779808+0000","last_change":"2024-08-18T13:43:55.959839+0000","last_active":"2024-08-18T13:51:51.779808+0000","last_peered":"2024-08-18T13:51:51.779808+0000","last_clean":"2024-08-18T13:51:51.779808+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:51:51.779808+0000","last_undegraded":"2024-08-18T13:51:51.779808+0000","last_fullsized":"2024-08-18T13:51:51.779808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029546540000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"54'85","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755227+0000","last_change":"2024-08-18T13:43:55.555415+0000","last_active":"2024-08-18T13:51:51.755227+0000","last_peered":"2024-08-18T13:51:51.755227+0000","last_clean":"2024-08-18T13:51:51.755227+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:51:51.755227+0000","last_undegraded":"2024-08-18T13:51:51.755227+0000","last_fullsized":"2024-08-18T13:51:51.755227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055234810000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"54'92","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762023+0000","last_change":"2024-08-18T13:43:55.916616+0000","last_active":"2024-08-18T13:51:51.762023+0000","last_peered":"2024-08-18T13:51:51.762023+0000","last_clean":"2024-08-18T13:51:51.762023+0000","last_became_active":"2024-08-18T13:40:16.614889+0000","last_became_peered":"2024-08-18T13:40:16.614889+0000","last_unstale":"2024-08-18T13:51:51.762023+0000","last_undegraded":"2024-08-18T13:51:51.762023+0000","last_fullsized":"2024-08-18T13:51:51.762023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:37:46.326075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033397829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761272+0000","last_change":"2024-08-18T13:43:55.554199+0000","last_active":"2024-08-18T13:51:51.761272+0000","last_peered":"2024-08-18T13:51:51.761272+0000","last_clean":"2024-08-18T13:51:51.761272+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:51:51.761272+0000","last_undegraded":"2024-08-18T13:51:51.761272+0000","last_fullsized":"2024-08-18T13:51:51.761272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070000389999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"54'95","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.780212+0000","last_change":"2024-08-18T13:43:55.863255+0000","last_active":"2024-08-18T13:51:51.780212+0000","last_peered":"2024-08-18T13:51:51.780212+0000","last_clean":"2024-08-18T13:51:51.780212+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:51:51.780212+0000","last_undegraded":"2024-08-18T13:51:51.780212+0000","last_fullsized":"2024-08-18T13:51:51.780212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070451519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"54'88","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778638+0000","last_change":"2024-08-18T13:43:56.152879+0000","last_active":"2024-08-18T13:51:51.778638+0000","last_peered":"2024-08-18T13:51:51.778638+0000","last_clean":"2024-08-18T13:51:51.778638+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:51:51.778638+0000","last_undegraded":"2024-08-18T13:51:51.778638+0000","last_fullsized":"2024-08-18T13:51:51.778638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070254549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.759908+0000","last_change":"2024-08-18T13:43:55.956991+0000","last_active":"2024-08-18T13:51:51.759908+0000","last_peered":"2024-08-18T13:51:51.759908+0000","last_clean":"2024-08-18T13:51:51.759908+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:51:51.759908+0000","last_undegraded":"2024-08-18T13:51:51.759908+0000","last_fullsized":"2024-08-18T13:51:51.759908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003261548,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"54'85","reported_seq":2143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758255+0000","last_change":"2024-08-18T13:43:56.141939+0000","last_active":"2024-08-18T13:51:51.758255+0000","last_peered":"2024-08-18T13:51:51.758255+0000","last_clean":"2024-08-18T13:51:51.758255+0000","last_became_active":"2024-08-18T13:40:16.612760+0000","last_became_peered":"2024-08-18T13:40:16.612760+0000","last_unstale":"2024-08-18T13:51:51.758255+0000","last_undegraded":"2024-08-18T13:51:51.758255+0000","last_fullsized":"2024-08-18T13:51:51.758255+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:58:42.688761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076223990000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767420+0000","last_change":"2024-08-18T13:43:55.917355+0000","last_active":"2024-08-18T13:51:51.767420+0000","last_peered":"2024-08-18T13:51:51.767420+0000","last_clean":"2024-08-18T13:51:51.767420+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:51:51.767420+0000","last_undegraded":"2024-08-18T13:51:51.767420+0000","last_fullsized":"2024-08-18T13:51:51.767420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027557649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"54'94","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779547+0000","last_change":"2024-08-18T13:43:55.860909+0000","last_active":"2024-08-18T13:51:51.779547+0000","last_peered":"2024-08-18T13:51:51.779547+0000","last_clean":"2024-08-18T13:51:51.779547+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:51:51.779547+0000","last_undegraded":"2024-08-18T13:51:51.779547+0000","last_fullsized":"2024-08-18T13:51:51.779547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069473399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"54'96","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775325+0000","last_change":"2024-08-18T13:43:55.860127+0000","last_active":"2024-08-18T13:51:51.775325+0000","last_peered":"2024-08-18T13:51:51.775325+0000","last_clean":"2024-08-18T13:51:51.775325+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:51:51.775325+0000","last_undegraded":"2024-08-18T13:51:51.775325+0000","last_fullsized":"2024-08-18T13:51:51.775325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070196240000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761102+0000","last_change":"2024-08-18T13:43:56.136762+0000","last_active":"2024-08-18T13:51:51.761102+0000","last_peered":"2024-08-18T13:51:51.761102+0000","last_clean":"2024-08-18T13:51:51.761102+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:51:51.761102+0000","last_undegraded":"2024-08-18T13:51:51.761102+0000","last_fullsized":"2024-08-18T13:51:51.761102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004509931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.760407+0000","last_change":"2024-08-18T13:43:56.154301+0000","last_active":"2024-08-18T13:51:51.760407+0000","last_peered":"2024-08-18T13:51:51.760407+0000","last_clean":"2024-08-18T13:51:51.760407+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:51:51.760407+0000","last_undegraded":"2024-08-18T13:51:51.760407+0000","last_fullsized":"2024-08-18T13:51:51.760407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062343609999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"54'81","reported_seq":2144,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767251+0000","last_change":"2024-08-18T13:43:55.554507+0000","last_active":"2024-08-18T13:51:51.767251+0000","last_peered":"2024-08-18T13:51:51.767251+0000","last_clean":"2024-08-18T13:51:51.767251+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:51:51.767251+0000","last_undegraded":"2024-08-18T13:51:51.767251+0000","last_fullsized":"2024-08-18T13:51:51.767251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072121939999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"54'77","reported_seq":2141,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767294+0000","last_change":"2024-08-18T13:43:55.558437+0000","last_active":"2024-08-18T13:51:51.767294+0000","last_peered":"2024-08-18T13:51:51.767294+0000","last_clean":"2024-08-18T13:51:51.767294+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:51:51.767294+0000","last_undegraded":"2024-08-18T13:51:51.767294+0000","last_fullsized":"2024-08-18T13:51:51.767294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057680220000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776234+0000","last_change":"2024-08-18T13:43:55.958601+0000","last_active":"2024-08-18T13:51:51.776234+0000","last_peered":"2024-08-18T13:51:51.776234+0000","last_clean":"2024-08-18T13:51:51.776234+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:51:51.776234+0000","last_undegraded":"2024-08-18T13:51:51.776234+0000","last_fullsized":"2024-08-18T13:51:51.776234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049025989999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779265+0000","last_change":"2024-08-18T13:43:55.959710+0000","last_active":"2024-08-18T13:51:51.779265+0000","last_peered":"2024-08-18T13:51:51.779265+0000","last_clean":"2024-08-18T13:51:51.779265+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:51:51.779265+0000","last_undegraded":"2024-08-18T13:51:51.779265+0000","last_fullsized":"2024-08-18T13:51:51.779265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029569129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"54'83","reported_seq":2146,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781497+0000","last_change":"2024-08-18T13:43:55.552920+0000","last_active":"2024-08-18T13:51:51.781497+0000","last_peered":"2024-08-18T13:51:51.781497+0000","last_clean":"2024-08-18T13:51:51.781497+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:51:51.781497+0000","last_undegraded":"2024-08-18T13:51:51.781497+0000","last_fullsized":"2024-08-18T13:51:51.781497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062586680000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"54'74","reported_seq":2137,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777962+0000","last_change":"2024-08-18T13:43:55.551745+0000","last_active":"2024-08-18T13:51:51.777962+0000","last_peered":"2024-08-18T13:51:51.777962+0000","last_clean":"2024-08-18T13:51:51.777962+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:51:51.777962+0000","last_undegraded":"2024-08-18T13:51:51.777962+0000","last_fullsized":"2024-08-18T13:51:51.777962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064056340000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"54'105","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774468+0000","last_change":"2024-08-18T13:43:55.554037+0000","last_active":"2024-08-18T13:51:51.774468+0000","last_peered":"2024-08-18T13:51:51.774468+0000","last_clean":"2024-08-18T13:51:51.774468+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:51:51.774468+0000","last_undegraded":"2024-08-18T13:51:51.774468+0000","last_fullsized":"2024-08-18T13:51:51.774468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050911749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"54'93","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774885+0000","last_change":"2024-08-18T13:43:55.862998+0000","last_active":"2024-08-18T13:51:51.774885+0000","last_peered":"2024-08-18T13:51:51.774885+0000","last_clean":"2024-08-18T13:51:51.774885+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:51:51.774885+0000","last_undegraded":"2024-08-18T13:51:51.774885+0000","last_fullsized":"2024-08-18T13:51:51.774885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068654359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"54'102","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776207+0000","last_change":"2024-08-18T13:43:55.550186+0000","last_active":"2024-08-18T13:51:51.776207+0000","last_peered":"2024-08-18T13:51:51.776207+0000","last_clean":"2024-08-18T13:51:51.776207+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:51:51.776207+0000","last_undegraded":"2024-08-18T13:51:51.776207+0000","last_fullsized":"2024-08-18T13:51:51.776207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055002549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"54'76","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774945+0000","last_change":"2024-08-18T13:43:55.859954+0000","last_active":"2024-08-18T13:51:51.774945+0000","last_peered":"2024-08-18T13:51:51.774945+0000","last_clean":"2024-08-18T13:51:51.774945+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:51:51.774945+0000","last_undegraded":"2024-08-18T13:51:51.774945+0000","last_fullsized":"2024-08-18T13:51:51.774945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069338170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762078+0000","last_change":"2024-08-18T13:43:56.150131+0000","last_active":"2024-08-18T13:51:51.762078+0000","last_peered":"2024-08-18T13:51:51.762078+0000","last_clean":"2024-08-18T13:51:51.762078+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:51:51.762078+0000","last_undegraded":"2024-08-18T13:51:51.762078+0000","last_fullsized":"2024-08-18T13:51:51.762078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044622389999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"54'76","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763798+0000","last_change":"2024-08-18T13:43:56.136824+0000","last_active":"2024-08-18T13:51:51.763798+0000","last_peered":"2024-08-18T13:51:51.763798+0000","last_clean":"2024-08-18T13:51:51.763798+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:51:51.763798+0000","last_undegraded":"2024-08-18T13:51:51.763798+0000","last_fullsized":"2024-08-18T13:51:51.763798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035965799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"54'86","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778822+0000","last_change":"2024-08-18T13:43:55.552484+0000","last_active":"2024-08-18T13:51:51.778822+0000","last_peered":"2024-08-18T13:51:51.778822+0000","last_clean":"2024-08-18T13:51:51.778822+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:51:51.778822+0000","last_undegraded":"2024-08-18T13:51:51.778822+0000","last_fullsized":"2024-08-18T13:51:51.778822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067690759999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":938443088,"num_objects":10906,"num_object_clones":0,"num_object_copies":43624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10906,"num_whiteouts":0,"num_read":21895,"num_read_kb":705026,"num_write":21484,"num_write_kb":1061752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2001649664,"data_stored":2001649664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5769,"ondisk_log_size":5769,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":184495130,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":586,"num_read_kb":685208,"num_write":6404,"num_write_kb":185940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":369049600,"data_stored":368990260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6329,"ondisk_log_size":6329,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599760,"kb_used_data":294088,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93772080,"statfs":{"total":96636764160,"available":96022609920,"internally_reserved":0,"allocated":301146112,"data_stored":300847166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2999999999999998,"5min":1.026,"15min":0.878},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":25.744,"5min":25.744,"15min":25.744},"last":1.254},{"interface":"front","average":{"1min":1.3149999999999999,"5min":0.84999999999999998,"15min":0.82599999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.33100000000000002},"max":{"1min":8.1679999999999993,"5min":8.1679999999999993,"15min":8.1679999999999993},"last":0.93799999999999994}]},{"osd":1,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":0.90700000000000003,"15min":0.98599999999999999},"min":{"1min":0.5,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":10.151999999999999,"5min":10.151999999999999,"15min":39.203000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":2.323,"5min":1.111,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":25.852,"5min":25.852,"15min":25.852},"last":0.96499999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":1.121,"15min":0.95099999999999996},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.39400000000000002},"max":{"1min":25.818999999999999,"5min":25.818999999999999,"15min":25.818999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.85199999999999998},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.35799999999999998},"max":{"1min":1.133,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.0840000000000001,"15min":1.135},"min":{"1min":0.64700000000000002,"5min":0.42699999999999999,"15min":0.377},"max":{"1min":17.300999999999998,"5min":17.300999999999998,"15min":42.905000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":2.4689999999999999,"5min":1.238,"15min":1.115},"min":{"1min":0.75600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":25.786999999999999,"5min":25.786999999999999,"15min":25.786999999999999},"last":1.323}]},{"osd":4,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.29699999999999999},"max":{"1min":8.3870000000000005,"5min":8.3870000000000005,"15min":8.3870000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.171,"15min":1.23},"min":{"1min":0.79800000000000004,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":17.056999999999999,"5min":17.056999999999999,"15min":52.673000000000002},"last":1.127}]},{"osd":5,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":1.274,"15min":1.119},"min":{"1min":0.44,"5min":0.44,"15min":0.33700000000000002},"max":{"1min":25.881,"5min":25.881,"15min":25.881},"last":1.087},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.1080000000000001,"15min":1.073},"min":{"1min":0.73899999999999999,"5min":0.54200000000000004,"15min":0.37},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":0.77100000000000002}]},{"osd":6,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.1659999999999999,"15min":1.1259999999999999},"min":{"1min":0.83799999999999997,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":8.0510000000000002,"5min":8.0510000000000002,"15min":8.0510000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.1990000000000001,"15min":1.26},"min":{"1min":0.85499999999999998,"5min":0.49399999999999999,"15min":0.435},"max":{"1min":10.523,"5min":10.523,"15min":39.161000000000001},"last":1.02}]}]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610072,"kb_used_data":312720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93761768,"statfs":{"total":96636764160,"available":96012050432,"internally_reserved":0,"allocated":320225280,"data_stored":319926507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76500000000000001,"15min":0.88},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.165,"5min":1.21,"15min":4.532},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.248},"max":{"1min":1.258,"5min":1.3200000000000001,"15min":1.9650000000000001},"last":0.73899999999999999}]},{"osd":1,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.92000000000000004},"min":{"1min":0.42399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2749999999999999,"5min":1.294,"15min":4.468},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.91900000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.30499999999999999},"max":{"1min":1.1990000000000001,"5min":1.286,"15min":2.3620000000000001},"last":1.01}]},{"osd":2,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77700000000000002,"15min":0.83899999999999997},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.28000000000000003},"max":{"1min":1.085,"5min":1.177,"15min":1.413},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.97299999999999998},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1060000000000001,"5min":1.468,"15min":4.5},"last":0.69999999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88700000000000001,"15min":0.94199999999999995},"min":{"1min":0.60899999999999999,"5min":0.41999999999999998,"15min":0.35399999999999998},"max":{"1min":1.1919999999999999,"5min":1.276,"15min":4.3250000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.91600000000000004},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.32900000000000001},"max":{"1min":1.0920000000000001,"5min":1.306,"15min":1.8069999999999999},"last":0.94899999999999995}]},{"osd":4,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92700000000000005,"15min":1.0109999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.228,"5min":1.552,"15min":5.2549999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.7230000000000001,"15min":4.6600000000000001},"last":1.145}]},{"osd":5,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":1.0129999999999999},"min":{"1min":0.54900000000000004,"5min":0.45400000000000001,"15min":0.373},"max":{"1min":1.179,"5min":1.3160000000000001,"15min":4.4009999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.042,"15min":1.1499999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.272,"5min":1.6879999999999999,"15min":6.1260000000000003},"last":1.125}]},{"osd":7,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0669999999999999,"15min":1.111},"min":{"1min":0.67600000000000005,"5min":0.55500000000000005,"15min":0.40500000000000003},"max":{"1min":1.3009999999999999,"5min":1.4830000000000001,"15min":1.9299999999999999},"last":1.103},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0609999999999999,"15min":1.1080000000000001},"min":{"1min":0.67500000000000004,"5min":0.46300000000000002,"15min":0.40200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":2.1150000000000002},"last":1.2330000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509594,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570040,"kb_used_data":272688,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93801800,"statfs":{"total":96636764160,"available":96053043200,"internally_reserved":0,"allocated":279232512,"data_stored":278928482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.24299999999999999},"max":{"1min":1.1679999999999999,"5min":1.204,"15min":1.22},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":1.085,"5min":1.339,"15min":1.73},"last":0.84399999999999997}]},{"osd":1,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82499999999999996,"15min":0.86799999999999999},"min":{"1min":0.54100000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.2909999999999999,"5min":1.415,"15min":3.6299999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.36699999999999999,"15min":0.27600000000000002},"max":{"1min":1.0589999999999999,"5min":1.28,"15min":1.5169999999999999},"last":1.139}]},{"osd":2,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.85799999999999998},"min":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.5920000000000001},"last":0.77000000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92100000000000004,"15min":0.91800000000000004},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.309},"max":{"1min":1.111,"5min":1.387,"15min":1.45},"last":1.222},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.641,"15min":1.641},"last":1.111}]},{"osd":5,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.313},"max":{"1min":1.3640000000000001,"5min":1.575,"15min":3.7789999999999999},"last":1.081},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.998,"15min":1.0149999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3129999999999999,"5min":1.754,"15min":1.754},"last":0.99399999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0580000000000001,"15min":1.079},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.504},"max":{"1min":1.6259999999999999,"5min":1.901,"15min":7.8490000000000002},"last":1.026},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.016,"15min":1.0629999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2809999999999999,"5min":1.5229999999999999,"15min":6.3380000000000001},"last":1.282}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40000000000000002},"max":{"1min":1.2470000000000001,"5min":1.7470000000000001,"15min":1.8400000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.151,"15min":1.1819999999999999},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.42399999999999999},"max":{"1min":1.6850000000000001,"5min":2.4569999999999999,"15min":3.7410000000000001},"last":1.165}]}]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683436,"kb_used_data":373604,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93688404,"statfs":{"total":96636764160,"available":95936925696,"internally_reserved":0,"allocated":382570496,"data_stored":382269906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.29499999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.89500000000000002,"5min":1.2050000000000001,"15min":1.2170000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67800000000000005,"15min":0.69299999999999995},"min":{"1min":0.44900000000000001,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":1.413,"15min":1.413},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.68600000000000005},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86499999999999999,"5min":1.46,"15min":3.4820000000000002},"last":1.089},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.751},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.8979999999999999},"last":0.96199999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.48599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1910000000000001,"5min":1.6419999999999999,"15min":2.415},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.87,"15min":0.86899999999999999},"min":{"1min":0.57799999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.2589999999999999,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.0580000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.173,"5min":1.681,"15min":3.2490000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.052},"min":{"1min":0.66300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5189999999999999,"5min":2.4870000000000001,"15min":3.8799999999999999},"last":1.2270000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.996,"15min":4.1059999999999999},"last":1.119},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0089999999999999,"15min":1.05},"min":{"1min":0.63200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1719999999999999,"5min":1.627,"15min":3.2799999999999998},"last":1.1459999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.504,"5min":0.504,"15min":0.41999999999999998},"max":{"1min":1.3480000000000001,"5min":1.833,"15min":2.1080000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.054,"15min":1.0880000000000001},"min":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.573,"15min":2.7469999999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4419999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.169},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.087,"15min":1.0960000000000001},"min":{"1min":0.71499999999999997,"5min":0.63200000000000001,"15min":0.51100000000000001},"max":{"1min":1.583,"5min":1.9039999999999999,"15min":2.1429999999999998},"last":1.337}]}]},{"osd":2,"up_from":14,"seq":60129542298,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530780,"kb_used_data":229268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93841060,"statfs":{"total":96636764160,"available":96093245440,"internally_reserved":0,"allocated":234770432,"data_stored":234477374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.186},"max":{"1min":1.0369999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":1.0960000000000001,"5min":3.27,"15min":3.27},"last":0.5}]},{"osd":1,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.038,"5min":1.1040000000000001,"15min":1.696},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":3.2290000000000001,"15min":3.2290000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.78200000000000003},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.253},"max":{"1min":0.995,"5min":1.077,"15min":12.468999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.84399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35899999999999999,"15min":0.27500000000000002},"max":{"1min":1.077,"5min":1.3540000000000001,"15min":1.744},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2709999999999999,"5min":1.681,"15min":1.8480000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.94799999999999995},"min":{"1min":0.46899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2609999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1759999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39000000000000001},"max":{"1min":1.149,"5min":1.498,"15min":1.627},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96899999999999997,"15min":0.98799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.42199999999999999},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.649},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.0309999999999999,"15min":1.0649999999999999},"min":{"1min":0.67000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":1.5820000000000001,"15min":1.905},"last":0.878}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0549999999999999,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.60399999999999998,"15min":0.436},"max":{"1min":1.3140000000000001,"5min":1.5089999999999999,"15min":1.9490000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.0149999999999999},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.46400000000000002},"max":{"1min":1.343,"5min":1.4379999999999999,"15min":1.7669999999999999},"last":0.90200000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542298,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677976,"kb_used_data":376464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93693864,"statfs":{"total":96636764160,"available":95942516736,"internally_reserved":0,"allocated":385499136,"data_stored":385199863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.65200000000000002},"min":{"1min":0.28899999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.94899999999999995,"5min":1.3009999999999999,"15min":3.1160000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.72099999999999997},"min":{"1min":0.30599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.107,"5min":1.327,"15min":3.718},"last":0.76900000000000002}]},{"osd":2,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0549999999999999,"5min":4.968,"15min":4.968},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33400000000000002,"15min":0.20899999999999999},"max":{"1min":1.327,"5min":1.3560000000000001,"15min":9.3309999999999995},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4810000000000001,"15min":1.927},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79400000000000004,"15min":0.84899999999999998},"min":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":1.46,"5min":1.46,"15min":6.8559999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.28000000000000003},"max":{"1min":1.488,"5min":5.1020000000000003,"15min":5.6529999999999996},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.63400000000000001,"5min":0.39700000000000002,"15min":0.28599999999999998},"max":{"1min":1.4410000000000001,"5min":1.554,"15min":1.5740000000000001},"last":1.125}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3859999999999999,"5min":1.546,"15min":3.7570000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1,"5min":1.04,"15min":1.1040000000000001},"min":{"1min":0.47799999999999998,"5min":0.46400000000000002,"15min":0.45700000000000002},"max":{"1min":1.591,"5min":5.1799999999999997,"15min":19.027999999999999},"last":1.272}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.40699999999999997},"max":{"1min":1.5529999999999999,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.113,"5min":1.0880000000000001,"15min":1.131},"min":{"1min":0.54900000000000004,"5min":0.49099999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.861},"last":0.98799999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.069,"15min":1.119},"min":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58799999999999997},"max":{"1min":1.383,"5min":1.526,"15min":3.8100000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40000000000000002},"max":{"1min":1.431,"5min":5.3099999999999996,"15min":5.3099999999999996},"last":1.3220000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542298,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549780,"kb_used_data":248268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93822060,"statfs":{"total":96636764160,"available":96073789440,"internally_reserved":0,"allocated":254226432,"data_stored":253919977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.65400000000000003},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":5.3159999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.33500000000000002,"5min":0.254,"15min":0.19500000000000001},"max":{"1min":0.93600000000000005,"5min":1.2370000000000001,"15min":3.4809999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.47099999999999997,"5min":0.32600000000000001,"15min":0.26300000000000001},"max":{"1min":0.99199999999999999,"5min":1.222,"15min":3.3029999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.28},"last":0.90000000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.288,"5min":1.288,"15min":5.399},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86799999999999999,"15min":0.93100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.33900000000000002},"max":{"1min":1.1799999999999999,"5min":1.365,"15min":5.0359999999999996},"last":0.99099999999999999}]},{"osd":4,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.37},"max":{"1min":1.3899999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":8.7799999999999994},"last":0.93999999999999995}]},{"osd":5,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96099999999999997,"15min":0.97499999999999998},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.377},{"interface":"front","average":{"1min":1.087,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.40300000000000002},"max":{"1min":1.51,"5min":1.52,"15min":1.8879999999999999},"last":0.82499999999999996}]},{"osd":6,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.77800000000000002,"5min":0.46500000000000002,"15min":0.374},"max":{"1min":1.423,"5min":1.599,"15min":3.7200000000000002},"last":1.171},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.042,"15min":1.085},"min":{"1min":0.68600000000000005,"5min":0.58099999999999996,"15min":0.52600000000000002},"max":{"1min":1.55,"5min":1.659,"15min":3.2789999999999999},"last":1.4239999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.143,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.60599999999999998,"15min":0.40300000000000002},"max":{"1min":1.514,"5min":1.704,"15min":36.295000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.171,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.55300000000000005,"15min":0.42999999999999999},"max":{"1min":1.5740000000000001,"5min":1.706,"15min":3.448},"last":1.206}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515540,"kb_used_data":214028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93856300,"statfs":{"total":96636764160,"available":96108851200,"internally_reserved":0,"allocated":219164672,"data_stored":218869425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.81100000000000005},"min":{"1min":0.48199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.135,"5min":1.2789999999999999,"15min":1.901},"last":0.747},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.81000000000000005,"15min":0.85599999999999998},"min":{"1min":0.63900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":4.0810000000000004},"last":0.88100000000000001}]},{"osd":1,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.82099999999999995},"min":{"1min":0.51900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.101,"5min":1.429,"15min":2.4809999999999999},"last":1.022},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1799999999999999,"5min":1.456,"15min":1.7749999999999999},"last":0.93400000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.63200000000000001,"5min":0.46999999999999997,"15min":0.41799999999999998},"max":{"1min":1.1419999999999999,"5min":1.524,"15min":1.774},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.874,"15min":0.93899999999999995},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.32500000000000001},"max":{"1min":1.1299999999999999,"5min":1.3160000000000001,"15min":4.1559999999999997},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.94399999999999995,"15min":0.97999999999999998},"min":{"1min":0.59799999999999998,"5min":0.54600000000000004,"15min":0.36199999999999999},"max":{"1min":1.3300000000000001,"5min":1.4410000000000001,"15min":4.5510000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.92700000000000005,"15min":0.98499999999999999},"min":{"1min":0.72199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.423,"5min":1.4470000000000001,"15min":3.976},"last":1.143}]},{"osd":4,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.7909999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0660000000000001,"15min":1.1120000000000001},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.6839999999999999,"15min":2.532},"last":1.2430000000000001}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.03,"15min":1.085},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3109999999999999,"5min":1.532,"15min":4.2050000000000001},"last":1.071},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0569999999999999,"15min":1.113},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":2.911},"last":1.2150000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.089,"15min":1.165},"min":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2989999999999999,"5min":1.659,"15min":2.569},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.1359999999999999,"15min":1.361},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40400000000000003},"max":{"1min":1.407,"5min":1.605,"15min":58.290999999999997},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41955328,"data_stored":41943666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50331808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21549056,"data_stored":21547239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34140160,"data_stored":34130811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53907456,"data_stored":53894667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41947136,"data_stored":41943130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50332308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":74870784,"data_stored":74866631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211058688,"data_stored":211058688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334540800,"data_stored":334540800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212602880,"data_stored":212602880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347811840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":224706560,"data_stored":224706560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176599040,"data_stored":176599040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":269266944,"data_stored":269266944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":225062912,"data_stored":225062912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:51:58.287 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-18T13:51:58.287 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:51:58.390 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81b06700 1 Processor -- start 2024-08-18T13:51:58.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81b06700 1 -- start start 2024-08-18T13:51:58.393 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81b06700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81b06700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c06f340 con 0x7fbd7c0ae970 2024-08-18T13:51:58.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81b06700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c0aeeb0 con 0x7fbd7c0add40 2024-08-18T13:51:58.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81b06700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c155110 con 0x7fbd7c0b15f0 2024-08-18T13:51:58.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81305700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbd7c0ae970 0x7fbd7c15fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38246/0 (socket says 172.21.15.118:38246) 2024-08-18T13:51:58.394 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd81305700 1 -- 172.21.15.118:0/820501942 learned_addr learned my addr 172.21.15.118:0/820501942 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7bfff700 1 --2- 172.21.15.118:0/820501942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7b7fe700 1 -- 172.21.15.118:0/820501942 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727437255 0 0) 0x7fbd7c0aeeb0 con 0x7fbd7c0add40 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7b7fe700 1 -- 172.21.15.118:0/820501942 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd68003570 con 0x7fbd7c0add40 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7b7fe700 1 -- 172.21.15.118:0/820501942 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885573050 0 0) 0x7fbd7c06f340 con 0x7fbd7c0ae970 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7b7fe700 1 -- 172.21.15.118:0/820501942 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd7c0aeeb0 con 0x7fbd7c0ae970 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7bfff700 1 -- 172.21.15.118:0/820501942 >> v1:172.21.15.178:6789/0 conn(0x7fbd7c0add40 legacy=0x7fbd7c155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7bfff700 1 -- 172.21.15.118:0/820501942 >> v1:172.21.15.118:6789/0 conn(0x7fbd7c0ae970 legacy=0x7fbd7c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.395 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7bfff700 1 -- 172.21.15.118:0/820501942 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0af150 con 0x7fbd7c0b15f0 2024-08-18T13:51:58.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.386+0000 7fbd7bfff700 1 --2- 172.21.15.118:0/820501942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c7500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009910 tx=0x7fbd640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8e21693c306d01 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd7b7fe700 1 -- 172.21.15.118:0/820501942 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400b690 con 0x7fbd7c0b15f0 2024-08-18T13:51:58.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd7b7fe700 1 -- 172.21.15.118:0/820501942 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6400b7f0 con 0x7fbd7c0b15f0 2024-08-18T13:51:58.396 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd7b7fe700 1 -- 172.21.15.118:0/820501942 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400bb90 con 0x7fbd7c0b15f0 2024-08-18T13:51:58.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 -- 172.21.15.118:0/820501942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0b15f0 msgr2=0x7fbd7c0c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 --2- 172.21.15.118:0/820501942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c7500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009910 tx=0x7fbd640091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 -- 172.21.15.118:0/820501942 shutdown_connections 2024-08-18T13:51:58.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 --2- 172.21.15.118:0/820501942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c7500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.399 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 -- 172.21.15.118:0/820501942 >> 172.21.15.118:0/820501942 conn(0x7fbd7c0cc600 msgr2=0x7fbd7c0c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 -- 172.21.15.118:0/820501942 shutdown_connections 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 -- 172.21.15.118:0/820501942 wait complete. 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 Processor -- start 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.390+0000 7fbd81b06700 1 -- start start 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd7c0add40 0x7fbd7c0c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0ae970 0x7fbd7c0c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.400 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c164ed0 con 0x7fbd7c0add40 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0c2e40 con 0x7fbd7c0b15f0 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0c36e0 con 0x7fbd7c0ae970 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd7c0add40 0x7fbd7c0c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd7bfff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0ae970 0x7fbd7c0c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd7c0add40 0x7fbd7c0c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44946/0 (socket says 172.21.15.118:44946) 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 -- 172.21.15.118:0/1482853024 learned_addr learned my addr 172.21.15.118:0/1482853024 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81305700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.401 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 -- 172.21.15.118:0/1482853024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0ae970 msgr2=0x7fbd7c0c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0ae970 0x7fbd7c0c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 -- 172.21.15.118:0/1482853024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c0b15f0 msgr2=0x7fbd7c0c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 -- 172.21.15.118:0/1482853024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009420 con 0x7fbd7c0add40 2024-08-18T13:51:58.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd80b04700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd7c0add40 0x7fbd7c0c3b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c7f0 tx=0x7fbd6c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd78ff9700 1 -- 172.21.15.118:0/1482853024 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0015b0 con 0x7fbd7c0add40 2024-08-18T13:51:58.402 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd78ff9700 1 -- 172.21.15.118:0/1482853024 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c001710 con 0x7fbd7c0add40 2024-08-18T13:51:58.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c0bf1a0 con 0x7fbd7c0add40 2024-08-18T13:51:58.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd78ff9700 1 -- 172.21.15.118:0/1482853024 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c001ad0 con 0x7fbd7c0add40 2024-08-18T13:51:58.403 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.394+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c1650e0 con 0x7fbd7c0add40 2024-08-18T13:51:58.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.398+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c0c7ed0 con 0x7fbd7c0add40 2024-08-18T13:51:58.404 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.398+0000 7fbd78ff9700 1 -- 172.21.15.118:0/1482853024 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbd6c01b400 con 0x7fbd7c0add40 2024-08-18T13:51:58.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.398+0000 7fbd78ff9700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd5c060860 0x7fbd5c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.398+0000 7fbd78ff9700 1 -- 172.21.15.118:0/1482853024 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fbd6c07cea0 con 0x7fbd7c0add40 2024-08-18T13:51:58.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.398+0000 7fbd7bfff700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd5c060860 0x7fbd5c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.405 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.398+0000 7fbd7bfff700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd5c060860 0x7fbd5c062d10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009860 tx=0x7fbd64017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.407 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.402+0000 7fbd78ff9700 1 -- 172.21.15.118:0/1482853024 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbd7c0c7ed0 con 0x7fbd7c0add40 2024-08-18T13:51:58.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.562+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd7c0c3820 con 0x7fbd5c060860 2024-08-18T13:51:58.586 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.582+0000 7fbd78ff9700 1 -- 172.21.15.118:0/1482853024 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+372617 (secure 0 0 0) 0x7fbd64020040 con 0x7fbd5c060860 2024-08-18T13:51:58.592 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:58.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd5c060860 msgr2=0x7fbd5c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd5c060860 0x7fbd5c062d10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009860 tx=0x7fbd64017000 comp rx=0 tx=0).stop 2024-08-18T13:51:58.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd7c0add40 msgr2=0x7fbd7c0c3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd7c0add40 0x7fbd7c0c3b70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c7f0 tx=0x7fbd6c007e30 comp rx=0 tx=0).stop 2024-08-18T13:51:58.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 shutdown_connections 2024-08-18T13:51:58.596 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd5c060860 0x7fbd5c062d10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd7c0b15f0 0x7fbd7c0c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd7c0ae970 0x7fbd7c0c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 --2- 172.21.15.118:0/1482853024 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd7c0add40 0x7fbd7c0c3b70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 >> 172.21.15.118:0/1482853024 conn(0x7fbd7c0cc600 msgr2=0x7fbd7c0c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:58.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 shutdown_connections 2024-08-18T13:51:58.597 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.586+0000 7fbd81b06700 1 -- 172.21.15.118:0/1482853024 wait complete. 2024-08-18T13:51:58.597 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:51:58.605 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-08-18T13:51:57.487883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1123528586,"num_objects":11047,"num_object_clones":0,"num_object_copies":43906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11047,"num_whiteouts":0,"num_read":22597,"num_read_kb":1390334,"num_write":28047,"num_write_kb":1250108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33741,"ondisk_log_size":33741,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4737384,"kb_used_data":2321128,"kb_used_omap":13,"kb_used_meta":2416178,"kb_avail":750237336,"statfs":{"total":773094113280,"available":768243032064,"internally_reserved":0,"allocated":2376835072,"data_stored":2374438700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2474166976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27132278,"num_objects":1405,"num_object_clones":0,"num_object_copies":5592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":9,"num_read_kb":916,"num_write":1574,"num_write_kb":27598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004102"},"pg_stats":[{"pgid":"2.1f","version":"53'151","reported_seq":214,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.743915+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:50:57.743915+0000","last_peered":"2024-08-18T13:50:57.743915+0000","last_clean":"2024-08-18T13:50:57.743915+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:50:57.743915+0000","last_undegraded":"2024-08-18T13:50:57.743915+0000","last_fullsized":"2024-08-18T13:50:57.743915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389178,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":28696,"num_write":150,"num_write_kb":8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766930+0000","last_change":"2024-08-18T13:43:55.555634+0000","last_active":"2024-08-18T13:51:51.766930+0000","last_peered":"2024-08-18T13:51:51.766930+0000","last_clean":"2024-08-18T13:51:51.766930+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:51:51.766930+0000","last_undegraded":"2024-08-18T13:51:51.766930+0000","last_fullsized":"2024-08-18T13:51:51.766930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064850389999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"54'676","reported_seq":3751,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.726737+0000","last_change":"2024-08-18T13:43:55.552107+0000","last_active":"2024-08-18T13:51:49.726737+0000","last_peered":"2024-08-18T13:51:49.726737+0000","last_clean":"2024-08-18T13:51:49.726737+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:51:49.726737+0000","last_undegraded":"2024-08-18T13:51:49.726737+0000","last_fullsized":"2024-08-18T13:51:49.726737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067303959999999996,"stat_sum":{"num_bytes":37566158,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":1013,"num_read_kb":36248,"num_write":673,"num_write_kb":41139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"54'78","reported_seq":126,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.670265+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:50:55.670265+0000","last_peered":"2024-08-18T13:50:55.670265+0000","last_clean":"2024-08-18T13:50:55.670265+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:50:55.670265+0000","last_undegraded":"2024-08-18T13:50:55.670265+0000","last_fullsized":"2024-08-18T13:50:55.670265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":75,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777966+0000","last_change":"2024-08-18T13:43:56.153208+0000","last_active":"2024-08-18T13:51:51.777966+0000","last_peered":"2024-08-18T13:51:51.777966+0000","last_clean":"2024-08-18T13:51:51.777966+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:51:51.777966+0000","last_undegraded":"2024-08-18T13:51:51.777966+0000","last_fullsized":"2024-08-18T13:51:51.777966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075430439999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"54'705","reported_seq":3559,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.547567+0000","last_change":"2024-08-18T13:43:55.555148+0000","last_active":"2024-08-18T13:51:49.547567+0000","last_peered":"2024-08-18T13:51:49.547567+0000","last_clean":"2024-08-18T13:51:49.547567+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:51:49.547567+0000","last_undegraded":"2024-08-18T13:51:49.547567+0000","last_fullsized":"2024-08-18T13:51:49.547567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:10:48.402406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075371290000000001,"stat_sum":{"num_bytes":29851183,"num_objects":370,"num_object_clones":0,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":792,"num_read_kb":20881,"num_write":708,"num_write_kb":37566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"54'194","reported_seq":259,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.297066+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:51:47.297066+0000","last_peered":"2024-08-18T13:51:47.297066+0000","last_clean":"2024-08-18T13:51:47.297066+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:51:47.297066+0000","last_undegraded":"2024-08-18T13:51:47.297066+0000","last_fullsized":"2024-08-18T13:51:47.297066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":69650,"num_write":192,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775007+0000","last_change":"2024-08-18T13:43:55.857350+0000","last_active":"2024-08-18T13:51:51.775007+0000","last_peered":"2024-08-18T13:51:51.775007+0000","last_clean":"2024-08-18T13:51:51.775007+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:51:51.775007+0000","last_undegraded":"2024-08-18T13:51:51.775007+0000","last_fullsized":"2024-08-18T13:51:51.775007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046085359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"54'680","reported_seq":3480,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.425160+0000","last_change":"2024-08-18T13:43:55.557895+0000","last_active":"2024-08-18T13:51:49.425160+0000","last_peered":"2024-08-18T13:51:49.425160+0000","last_clean":"2024-08-18T13:51:49.425160+0000","last_became_active":"2024-08-18T13:40:16.608743+0000","last_became_peered":"2024-08-18T13:40:16.608743+0000","last_unstale":"2024-08-18T13:51:49.425160+0000","last_undegraded":"2024-08-18T13:51:49.425160+0000","last_fullsized":"2024-08-18T13:51:49.425160+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:59:49.527489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080275980000000004,"stat_sum":{"num_bytes":28774140,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":727,"num_read_kb":13856,"num_write":680,"num_write_kb":28426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"53'23","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.459383+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:50:56.459383+0000","last_peered":"2024-08-18T13:50:56.459383+0000","last_clean":"2024-08-18T13:50:56.459383+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:50:56.459383+0000","last_undegraded":"2024-08-18T13:50:56.459383+0000","last_fullsized":"2024-08-18T13:50:56.459383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":25,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777014+0000","last_change":"2024-08-18T13:43:55.862914+0000","last_active":"2024-08-18T13:51:51.777014+0000","last_peered":"2024-08-18T13:51:51.777014+0000","last_clean":"2024-08-18T13:51:51.777014+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:51:51.777014+0000","last_undegraded":"2024-08-18T13:51:51.777014+0000","last_fullsized":"2024-08-18T13:51:51.777014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065299970000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"54'674","reported_seq":2902,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.448739+0000","last_change":"2024-08-18T13:43:55.555499+0000","last_active":"2024-08-18T13:51:49.448739+0000","last_peered":"2024-08-18T13:51:49.448739+0000","last_clean":"2024-08-18T13:51:49.448739+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:51:49.448739+0000","last_undegraded":"2024-08-18T13:51:49.448739+0000","last_fullsized":"2024-08-18T13:51:49.448739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:36:23.382334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078514629999999995,"stat_sum":{"num_bytes":11851705,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":166,"num_read_kb":1355,"num_write":673,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755374+0000","last_change":"2024-08-18T13:43:55.954106+0000","last_active":"2024-08-18T13:51:51.755374+0000","last_peered":"2024-08-18T13:51:51.755374+0000","last_clean":"2024-08-18T13:51:51.755374+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:51:51.755374+0000","last_undegraded":"2024-08-18T13:51:51.755374+0000","last_fullsized":"2024-08-18T13:51:51.755374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033699279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"54'215","reported_seq":274,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:53.770156+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:51:53.770156+0000","last_peered":"2024-08-18T13:51:53.770156+0000","last_clean":"2024-08-18T13:51:53.770156+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:51:53.770156+0000","last_undegraded":"2024-08-18T13:51:53.770156+0000","last_fullsized":"2024-08-18T13:51:53.770156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11951501,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":23,"num_read_kb":45068,"num_write":210,"num_write_kb":11823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"54'638","reported_seq":3281,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.214771+0000","last_change":"2024-08-18T13:43:55.558893+0000","last_active":"2024-08-18T13:51:49.214771+0000","last_peered":"2024-08-18T13:51:49.214771+0000","last_clean":"2024-08-18T13:51:49.214771+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:51:49.214771+0000","last_undegraded":"2024-08-18T13:51:49.214771+0000","last_fullsized":"2024-08-18T13:51:49.214771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:33:00.943522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059019540000000001,"stat_sum":{"num_bytes":16782769,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":581,"num_read_kb":14450,"num_write":639,"num_write_kb":24822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763887+0000","last_change":"2024-08-18T13:43:55.554204+0000","last_active":"2024-08-18T13:51:51.763887+0000","last_peered":"2024-08-18T13:51:51.763887+0000","last_clean":"2024-08-18T13:51:51.763887+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:51:51.763887+0000","last_undegraded":"2024-08-18T13:51:51.763887+0000","last_fullsized":"2024-08-18T13:51:51.763887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074252320000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"54'28","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.687997+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:50:56.687997+0000","last_peered":"2024-08-18T13:50:56.687997+0000","last_clean":"2024-08-18T13:50:56.687997+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:50:56.687997+0000","last_undegraded":"2024-08-18T13:50:56.687997+0000","last_fullsized":"2024-08-18T13:50:56.687997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":28,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"54'709","reported_seq":3799,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.722324+0000","last_change":"2024-08-18T13:43:55.558557+0000","last_active":"2024-08-18T13:51:49.722324+0000","last_peered":"2024-08-18T13:51:49.722324+0000","last_clean":"2024-08-18T13:51:49.722324+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:51:49.722324+0000","last_undegraded":"2024-08-18T13:51:49.722324+0000","last_fullsized":"2024-08-18T13:51:49.722324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:58:55.964858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057029610000000003,"stat_sum":{"num_bytes":33413248,"num_objects":347,"num_object_clones":0,"num_object_copies":1388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1028,"num_read_kb":27410,"num_write":707,"num_write_kb":41185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761963+0000","last_change":"2024-08-18T13:43:55.553624+0000","last_active":"2024-08-18T13:51:51.761963+0000","last_peered":"2024-08-18T13:51:51.761963+0000","last_clean":"2024-08-18T13:51:51.761963+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:51:51.761963+0000","last_undegraded":"2024-08-18T13:51:51.761963+0000","last_fullsized":"2024-08-18T13:51:51.761963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069243719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"53'13","reported_seq":57,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:50.648992+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:50:50.648992+0000","last_peered":"2024-08-18T13:50:50.648992+0000","last_clean":"2024-08-18T13:50:50.648992+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:50:50.648992+0000","last_undegraded":"2024-08-18T13:50:50.648992+0000","last_fullsized":"2024-08-18T13:50:50.648992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:28.804474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":11,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'683","reported_seq":3570,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.650342+0000","last_change":"2024-08-18T13:43:56.142589+0000","last_active":"2024-08-18T13:51:49.650342+0000","last_peered":"2024-08-18T13:51:49.650342+0000","last_clean":"2024-08-18T13:51:49.650342+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:51:49.650342+0000","last_undegraded":"2024-08-18T13:51:49.650342+0000","last_fullsized":"2024-08-18T13:51:49.650342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:53.526616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079287170000000001,"stat_sum":{"num_bytes":20263748,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":825,"num_read_kb":25932,"num_write":679,"num_write_kb":28331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:50.650367+0000","last_change":"2024-08-18T13:43:55.558187+0000","last_active":"2024-08-18T13:50:50.650367+0000","last_peered":"2024-08-18T13:50:50.650367+0000","last_clean":"2024-08-18T13:50:50.650367+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:50:50.650367+0000","last_undegraded":"2024-08-18T13:50:50.650367+0000","last_fullsized":"2024-08-18T13:50:50.650367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:52.758930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068791520000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"54'106","reported_seq":153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.256673+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:51:41.256673+0000","last_peered":"2024-08-18T13:51:41.256673+0000","last_clean":"2024-08-18T13:51:41.256673+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:51:41.256673+0000","last_undegraded":"2024-08-18T13:51:41.256673+0000","last_fullsized":"2024-08-18T13:51:41.256673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":108,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"54'617","reported_seq":3452,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.520510+0000","last_change":"2024-08-18T13:43:55.559559+0000","last_active":"2024-08-18T13:51:49.520510+0000","last_peered":"2024-08-18T13:51:49.520510+0000","last_clean":"2024-08-18T13:51:49.520510+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:51:49.520510+0000","last_undegraded":"2024-08-18T13:51:49.520510+0000","last_fullsized":"2024-08-18T13:51:49.520510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:38:51.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004078028,"stat_sum":{"num_bytes":24272349,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":773,"num_read_kb":25488,"num_write":617,"num_write_kb":28093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'93","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762461+0000","last_change":"2024-08-18T13:43:56.154543+0000","last_active":"2024-08-18T13:51:51.762461+0000","last_peered":"2024-08-18T13:51:51.762461+0000","last_clean":"2024-08-18T13:51:51.762461+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:51:51.762461+0000","last_undegraded":"2024-08-18T13:51:51.762461+0000","last_fullsized":"2024-08-18T13:51:51.762461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:44.729558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060661889999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"54'111","reported_seq":160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.305665+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:51:17.305665+0000","last_peered":"2024-08-18T13:51:17.305665+0000","last_clean":"2024-08-18T13:51:17.305665+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:51:17.305665+0000","last_undegraded":"2024-08-18T13:51:17.305665+0000","last_fullsized":"2024-08-18T13:51:17.305665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":111,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"54'720","reported_seq":2969,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.297825+0000","last_change":"2024-08-18T13:43:55.557702+0000","last_active":"2024-08-18T13:51:49.297825+0000","last_peered":"2024-08-18T13:51:49.297825+0000","last_clean":"2024-08-18T13:51:49.297825+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:51:49.297825+0000","last_undegraded":"2024-08-18T13:51:49.297825+0000","last_fullsized":"2024-08-18T13:51:49.297825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:11:17.806836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079796720000000002,"stat_sum":{"num_bytes":17871290,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":187,"num_read_kb":7768,"num_write":717,"num_write_kb":21887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781486+0000","last_change":"2024-08-18T13:43:55.555491+0000","last_active":"2024-08-18T13:51:51.781486+0000","last_peered":"2024-08-18T13:51:51.781486+0000","last_clean":"2024-08-18T13:51:51.781486+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:51:51.781486+0000","last_undegraded":"2024-08-18T13:51:51.781486+0000","last_fullsized":"2024-08-18T13:51:51.781486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054340660000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"54'66","reported_seq":116,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:48.746876+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:51:48.746876+0000","last_peered":"2024-08-18T13:51:48.746876+0000","last_clean":"2024-08-18T13:51:48.746876+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:51:48.746876+0000","last_undegraded":"2024-08-18T13:51:48.746876+0000","last_fullsized":"2024-08-18T13:51:48.746876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":184,"num_write":164,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"54'714","reported_seq":2929,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.710846+0000","last_change":"2024-08-18T13:43:55.550748+0000","last_active":"2024-08-18T13:51:49.710846+0000","last_peered":"2024-08-18T13:51:49.710846+0000","last_clean":"2024-08-18T13:51:49.710846+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:51:49.710846+0000","last_undegraded":"2024-08-18T13:51:49.710846+0000","last_fullsized":"2024-08-18T13:51:49.710846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057306029999999999,"stat_sum":{"num_bytes":13242151,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":153,"num_read_kb":5341,"num_write":712,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758218+0000","last_change":"2024-08-18T13:43:56.143836+0000","last_active":"2024-08-18T13:51:51.758218+0000","last_peered":"2024-08-18T13:51:51.758218+0000","last_clean":"2024-08-18T13:51:51.758218+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:51:51.758218+0000","last_undegraded":"2024-08-18T13:51:51.758218+0000","last_fullsized":"2024-08-18T13:51:51.758218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:42:01.774788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064599770000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"54'289","reported_seq":337,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:33.863944+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:51:33.863944+0000","last_peered":"2024-08-18T13:51:33.863944+0000","last_clean":"2024-08-18T13:51:33.863944+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:51:33.863944+0000","last_undegraded":"2024-08-18T13:51:33.863944+0000","last_fullsized":"2024-08-18T13:51:33.863944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":12176,"num_write":284,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"54'704","reported_seq":3425,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.623916+0000","last_change":"2024-08-18T13:43:56.151855+0000","last_active":"2024-08-18T13:51:49.623916+0000","last_peered":"2024-08-18T13:51:49.623916+0000","last_clean":"2024-08-18T13:51:49.623916+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:51:49.623916+0000","last_undegraded":"2024-08-18T13:51:49.623916+0000","last_fullsized":"2024-08-18T13:51:49.623916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060004689999999996,"stat_sum":{"num_bytes":28446730,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":659,"num_read_kb":21792,"num_write":702,"num_write_kb":28155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"54'84","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775443+0000","last_change":"2024-08-18T13:43:55.863531+0000","last_active":"2024-08-18T13:51:51.775443+0000","last_peered":"2024-08-18T13:51:51.775443+0000","last_clean":"2024-08-18T13:51:51.775443+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:51:51.775443+0000","last_undegraded":"2024-08-18T13:51:51.775443+0000","last_fullsized":"2024-08-18T13:51:51.775443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006455484,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"54'657","reported_seq":700,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.994466+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:51:38.994466+0000","last_peered":"2024-08-18T13:51:38.994466+0000","last_clean":"2024-08-18T13:51:38.994466+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:51:38.994466+0000","last_undegraded":"2024-08-18T13:51:38.994466+0000","last_fullsized":"2024-08-18T13:51:38.994466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"54'691","reported_seq":3343,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.557816+0000","last_change":"2024-08-18T13:43:56.141004+0000","last_active":"2024-08-18T13:51:49.557816+0000","last_peered":"2024-08-18T13:51:49.557816+0000","last_clean":"2024-08-18T13:51:49.557816+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:51:49.557816+0000","last_undegraded":"2024-08-18T13:51:49.557816+0000","last_fullsized":"2024-08-18T13:51:49.557816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067583870000000002,"stat_sum":{"num_bytes":38683931,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":590,"num_read_kb":16518,"num_write":694,"num_write_kb":42167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"54'97","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758746+0000","last_change":"2024-08-18T13:43:55.559256+0000","last_active":"2024-08-18T13:51:51.758746+0000","last_peered":"2024-08-18T13:51:51.758746+0000","last_clean":"2024-08-18T13:51:51.758746+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:51:51.758746+0000","last_undegraded":"2024-08-18T13:51:51.758746+0000","last_fullsized":"2024-08-18T13:51:51.758746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:18:00.648981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005854756,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"53'383","reported_seq":438,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.690458+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:50:55.690458+0000","last_peered":"2024-08-18T13:50:55.690458+0000","last_clean":"2024-08-18T13:50:55.690458+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:50:55.690458+0000","last_undegraded":"2024-08-18T13:50:55.690458+0000","last_fullsized":"2024-08-18T13:50:55.690458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":43191,"num_write":382,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"54'661","reported_seq":4460,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.576473+0000","last_change":"2024-08-18T13:43:56.144223+0000","last_active":"2024-08-18T13:51:49.576473+0000","last_peered":"2024-08-18T13:51:49.576473+0000","last_clean":"2024-08-18T13:51:49.576473+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:51:49.576473+0000","last_undegraded":"2024-08-18T13:51:49.576473+0000","last_fullsized":"2024-08-18T13:51:49.576473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:09:14.840072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067969390000000001,"stat_sum":{"num_bytes":51061167,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":1737,"num_read_kb":43040,"num_write":661,"num_write_kb":53481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"54'83","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758102+0000","last_change":"2024-08-18T13:43:56.143663+0000","last_active":"2024-08-18T13:51:51.758102+0000","last_peered":"2024-08-18T13:51:51.758102+0000","last_clean":"2024-08-18T13:51:51.758102+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:51:51.758102+0000","last_undegraded":"2024-08-18T13:51:51.758102+0000","last_fullsized":"2024-08-18T13:51:51.758102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063915389999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"53'120","reported_seq":283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.353455+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:50:51.353455+0000","last_peered":"2024-08-18T13:50:51.353455+0000","last_clean":"2024-08-18T13:50:51.353455+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:50:51.353455+0000","last_undegraded":"2024-08-18T13:50:51.353455+0000","last_fullsized":"2024-08-18T13:50:51.353455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":638,"num_write":117,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"54'643","reported_seq":3597,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.312624+0000","last_change":"2024-08-18T13:43:56.143097+0000","last_active":"2024-08-18T13:51:49.312624+0000","last_peered":"2024-08-18T13:51:49.312624+0000","last_clean":"2024-08-18T13:51:49.312624+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:51:49.312624+0000","last_undegraded":"2024-08-18T13:51:49.312624+0000","last_fullsized":"2024-08-18T13:51:49.312624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079242040000000007,"stat_sum":{"num_bytes":41790406,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":892,"num_read_kb":23319,"num_write":639,"num_write_kb":45248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'82","reported_seq":2146,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762481+0000","last_change":"2024-08-18T13:43:56.149372+0000","last_active":"2024-08-18T13:51:51.762481+0000","last_peered":"2024-08-18T13:51:51.762481+0000","last_clean":"2024-08-18T13:51:51.762481+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:51:51.762481+0000","last_undegraded":"2024-08-18T13:51:51.762481+0000","last_fullsized":"2024-08-18T13:51:51.762481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041495459999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"54'234","reported_seq":290,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.626190+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:50:57.626190+0000","last_peered":"2024-08-18T13:50:57.626190+0000","last_clean":"2024-08-18T13:50:57.626190+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:50:57.626190+0000","last_undegraded":"2024-08-18T13:50:57.626190+0000","last_fullsized":"2024-08-18T13:50:57.626190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":49152,"num_write":231,"num_write_kb":12421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"54'716","reported_seq":3216,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.491284+0000","last_change":"2024-08-18T13:43:56.153929+0000","last_active":"2024-08-18T13:51:49.491284+0000","last_peered":"2024-08-18T13:51:49.491284+0000","last_clean":"2024-08-18T13:51:49.491284+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:51:49.491284+0000","last_undegraded":"2024-08-18T13:51:49.491284+0000","last_fullsized":"2024-08-18T13:51:49.491284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:49:26.256315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058974379999999996,"stat_sum":{"num_bytes":30462084,"num_objects":352,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":439,"num_read_kb":15189,"num_write":710,"num_write_kb":30067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779966+0000","last_change":"2024-08-18T13:43:55.552534+0000","last_active":"2024-08-18T13:51:51.779966+0000","last_peered":"2024-08-18T13:51:51.779966+0000","last_clean":"2024-08-18T13:51:51.779966+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:51:51.779966+0000","last_undegraded":"2024-08-18T13:51:51.779966+0000","last_fullsized":"2024-08-18T13:51:51.779966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068182370000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"54'982","reported_seq":1027,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.365615+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:51:17.365615+0000","last_peered":"2024-08-18T13:51:17.365615+0000","last_clean":"2024-08-18T13:51:17.365615+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:51:17.365615+0000","last_undegraded":"2024-08-18T13:51:17.365615+0000","last_fullsized":"2024-08-18T13:51:17.365615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2087,"num_write":979,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:49:30.943967+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:49:30.943967+0000","last_peered":"2024-08-18T13:49:30.943967+0000","last_clean":"2024-08-18T13:49:30.943967+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:49:30.943967+0000","last_undegraded":"2024-08-18T13:49:30.943967+0000","last_fullsized":"2024-08-18T13:49:30.943967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"54'664","reported_seq":3524,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.497597+0000","last_change":"2024-08-18T13:43:55.956673+0000","last_active":"2024-08-18T13:51:49.497597+0000","last_peered":"2024-08-18T13:51:49.497597+0000","last_clean":"2024-08-18T13:51:49.497597+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:51:49.497597+0000","last_undegraded":"2024-08-18T13:51:49.497597+0000","last_fullsized":"2024-08-18T13:51:49.497597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035684480000000001,"stat_sum":{"num_bytes":39930170,"num_objects":364,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":798,"num_read_kb":26403,"num_write":662,"num_write_kb":47533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779241+0000","last_change":"2024-08-18T13:43:55.552313+0000","last_active":"2024-08-18T13:51:51.779241+0000","last_peered":"2024-08-18T13:51:51.779241+0000","last_clean":"2024-08-18T13:51:51.779241+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:51:51.779241+0000","last_undegraded":"2024-08-18T13:51:51.779241+0000","last_fullsized":"2024-08-18T13:51:51.779241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057711639999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"54'76","reported_seq":118,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.291821+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:51:47.291821+0000","last_peered":"2024-08-18T13:51:47.291821+0000","last_clean":"2024-08-18T13:51:47.291821+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:51:47.291821+0000","last_undegraded":"2024-08-18T13:51:47.291821+0000","last_fullsized":"2024-08-18T13:51:47.291821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"54'688","reported_seq":4127,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.578882+0000","last_change":"2024-08-18T13:43:55.953251+0000","last_active":"2024-08-18T13:51:49.578882+0000","last_peered":"2024-08-18T13:51:49.578882+0000","last_clean":"2024-08-18T13:51:49.578882+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:51:49.578882+0000","last_undegraded":"2024-08-18T13:51:49.578882+0000","last_fullsized":"2024-08-18T13:51:49.578882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032433420000000002,"stat_sum":{"num_bytes":54557211,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1377,"num_read_kb":50055,"num_write":687,"num_write_kb":57734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"54'95","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.756325+0000","last_change":"2024-08-18T13:43:55.918416+0000","last_active":"2024-08-18T13:51:51.756325+0000","last_peered":"2024-08-18T13:51:51.756325+0000","last_clean":"2024-08-18T13:51:51.756325+0000","last_became_active":"2024-08-18T13:40:16.614814+0000","last_became_peered":"2024-08-18T13:40:16.614814+0000","last_unstale":"2024-08-18T13:51:51.756325+0000","last_undegraded":"2024-08-18T13:51:51.756325+0000","last_fullsized":"2024-08-18T13:51:51.756325+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:02:39.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033975450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'31","reported_seq":85,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.186584+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:50:56.186584+0000","last_peered":"2024-08-18T13:50:56.186584+0000","last_clean":"2024-08-18T13:50:56.186584+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:50:56.186584+0000","last_undegraded":"2024-08-18T13:50:56.186584+0000","last_fullsized":"2024-08-18T13:50:56.186584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":33,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"54'670","reported_seq":3234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.448131+0000","last_change":"2024-08-18T13:43:56.153281+0000","last_active":"2024-08-18T13:51:49.448131+0000","last_peered":"2024-08-18T13:51:49.448131+0000","last_clean":"2024-08-18T13:51:49.448131+0000","last_became_active":"2024-08-18T13:40:16.609908+0000","last_became_peered":"2024-08-18T13:40:16.609908+0000","last_unstale":"2024-08-18T13:51:49.448131+0000","last_undegraded":"2024-08-18T13:51:49.448131+0000","last_fullsized":"2024-08-18T13:51:49.448131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:35:52.383892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069871079999999997,"stat_sum":{"num_bytes":26719276,"num_objects":358,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":491,"num_read_kb":18148,"num_write":671,"num_write_kb":26414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4,5],"acting":[0,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784591+0000","last_change":"2024-08-18T13:43:55.553120+0000","last_active":"2024-08-18T13:51:51.784591+0000","last_peered":"2024-08-18T13:51:51.784591+0000","last_clean":"2024-08-18T13:51:51.784591+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:51:51.784591+0000","last_undegraded":"2024-08-18T13:51:51.784591+0000","last_fullsized":"2024-08-18T13:51:51.784591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057951929999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"53'29","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.394499+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:50:56.394499+0000","last_peered":"2024-08-18T13:50:56.394499+0000","last_clean":"2024-08-18T13:50:56.394499+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:50:56.394499+0000","last_undegraded":"2024-08-18T13:50:56.394499+0000","last_fullsized":"2024-08-18T13:50:56.394499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":30,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"54'688","reported_seq":2818,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.687818+0000","last_change":"2024-08-18T13:43:55.859666+0000","last_active":"2024-08-18T13:51:49.687818+0000","last_peered":"2024-08-18T13:51:49.687818+0000","last_clean":"2024-08-18T13:51:49.687818+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:51:49.687818+0000","last_undegraded":"2024-08-18T13:51:49.687818+0000","last_fullsized":"2024-08-18T13:51:49.687818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070733389999999997,"stat_sum":{"num_bytes":17282460,"num_objects":346,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":68,"num_read_kb":8224,"num_write":686,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"54'66","reported_seq":2129,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783626+0000","last_change":"2024-08-18T13:43:55.551809+0000","last_active":"2024-08-18T13:51:51.783626+0000","last_peered":"2024-08-18T13:51:51.783626+0000","last_clean":"2024-08-18T13:51:51.783626+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:51:51.783626+0000","last_undegraded":"2024-08-18T13:51:51.783626+0000","last_fullsized":"2024-08-18T13:51:51.783626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060906930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"53'26","reported_seq":80,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.647720+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:50:55.647720+0000","last_peered":"2024-08-18T13:50:55.647720+0000","last_clean":"2024-08-18T13:50:55.647720+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:50:55.647720+0000","last_undegraded":"2024-08-18T13:50:55.647720+0000","last_fullsized":"2024-08-18T13:50:55.647720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:07:43.092104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":27,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"54'695","reported_seq":3630,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.666776+0000","last_change":"2024-08-18T13:43:55.557389+0000","last_active":"2024-08-18T13:51:49.666776+0000","last_peered":"2024-08-18T13:51:49.666776+0000","last_clean":"2024-08-18T13:51:49.666776+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:51:49.666776+0000","last_undegraded":"2024-08-18T13:51:49.666776+0000","last_fullsized":"2024-08-18T13:51:49.666776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:03:22.905693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011606655,"stat_sum":{"num_bytes":44119789,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":873,"num_read_kb":37157,"num_write":696,"num_write_kb":47501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"54'91","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784041+0000","last_change":"2024-08-18T13:43:55.553915+0000","last_active":"2024-08-18T13:51:51.784041+0000","last_peered":"2024-08-18T13:51:51.784041+0000","last_clean":"2024-08-18T13:51:51.784041+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:51:51.784041+0000","last_undegraded":"2024-08-18T13:51:51.784041+0000","last_fullsized":"2024-08-18T13:51:51.784041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053019490000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"53'1204","reported_seq":1276,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.252643+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:51:17.252643+0000","last_peered":"2024-08-18T13:51:17.252643+0000","last_clean":"2024-08-18T13:51:17.252643+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:51:17.252643+0000","last_undegraded":"2024-08-18T13:51:17.252643+0000","last_fullsized":"2024-08-18T13:51:17.252643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":78,"num_read_kb":12399,"num_write":1202,"num_write_kb":8830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'657","reported_seq":3516,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.665369+0000","last_change":"2024-08-18T13:43:56.154837+0000","last_active":"2024-08-18T13:51:49.665369+0000","last_peered":"2024-08-18T13:51:49.665369+0000","last_clean":"2024-08-18T13:51:49.665369+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:51:49.665369+0000","last_undegraded":"2024-08-18T13:51:49.665369+0000","last_fullsized":"2024-08-18T13:51:49.665369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054104280000000001,"stat_sum":{"num_bytes":33280018,"num_objects":296,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296,"num_whiteouts":0,"num_read":797,"num_read_kb":22024,"num_write":655,"num_write_kb":32783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757105+0000","last_change":"2024-08-18T13:43:55.959459+0000","last_active":"2024-08-18T13:51:51.757105+0000","last_peered":"2024-08-18T13:51:51.757105+0000","last_clean":"2024-08-18T13:51:51.757105+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:51:51.757105+0000","last_undegraded":"2024-08-18T13:51:51.757105+0000","last_fullsized":"2024-08-18T13:51:51.757105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044255090000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"54'5","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.846481+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:50:51.846481+0000","last_peered":"2024-08-18T13:50:51.846481+0000","last_clean":"2024-08-18T13:50:51.846481+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:50:51.846481+0000","last_undegraded":"2024-08-18T13:50:51.846481+0000","last_fullsized":"2024-08-18T13:50:51.846481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"54'660","reported_seq":2779,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.587481+0000","last_change":"2024-08-18T13:43:55.555242+0000","last_active":"2024-08-18T13:51:49.587481+0000","last_peered":"2024-08-18T13:51:49.587481+0000","last_clean":"2024-08-18T13:51:49.587481+0000","last_became_active":"2024-08-18T13:40:16.915369+0000","last_became_peered":"2024-08-18T13:40:16.915369+0000","last_unstale":"2024-08-18T13:51:49.587481+0000","last_undegraded":"2024-08-18T13:51:49.587481+0000","last_fullsized":"2024-08-18T13:51:49.587481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:50:23.633304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005619059,"stat_sum":{"num_bytes":16977484,"num_objects":353,"num_object_clones":0,"num_object_copies":1412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":46,"num_read_kb":4277,"num_write":660,"num_write_kb":20975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,6],"acting":[3,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"54'101","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784667+0000","last_change":"2024-08-18T13:43:55.551514+0000","last_active":"2024-08-18T13:51:51.784667+0000","last_peered":"2024-08-18T13:51:51.784667+0000","last_clean":"2024-08-18T13:51:51.784667+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:51:51.784667+0000","last_undegraded":"2024-08-18T13:51:51.784667+0000","last_fullsized":"2024-08-18T13:51:51.784667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064019180000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"53'2","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:50:28.547615+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:50:28.547615+0000","last_peered":"2024-08-18T13:50:28.547615+0000","last_clean":"2024-08-18T13:50:28.547615+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:50:28.547615+0000","last_undegraded":"2024-08-18T13:50:28.547615+0000","last_fullsized":"2024-08-18T13:50:28.547615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"54'625","reported_seq":3598,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.686816+0000","last_change":"2024-08-18T13:43:55.554811+0000","last_active":"2024-08-18T13:51:49.686816+0000","last_peered":"2024-08-18T13:51:49.686816+0000","last_clean":"2024-08-18T13:51:49.686816+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:51:49.686816+0000","last_undegraded":"2024-08-18T13:51:49.686816+0000","last_fullsized":"2024-08-18T13:51:49.686816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074125900000000002,"stat_sum":{"num_bytes":40693231,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":911,"num_read_kb":37155,"num_write":622,"num_write_kb":48255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778223+0000","last_change":"2024-08-18T13:43:56.154386+0000","last_active":"2024-08-18T13:51:51.778223+0000","last_peered":"2024-08-18T13:51:51.778223+0000","last_clean":"2024-08-18T13:51:51.778223+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:51:51.778223+0000","last_undegraded":"2024-08-18T13:51:51.778223+0000","last_fullsized":"2024-08-18T13:51:51.778223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059765870000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"54'216","reported_seq":283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.281097+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:51:41.281097+0000","last_peered":"2024-08-18T13:51:41.281097+0000","last_clean":"2024-08-18T13:51:41.281097+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:51:41.281097+0000","last_undegraded":"2024-08-18T13:51:41.281097+0000","last_fullsized":"2024-08-18T13:51:41.281097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":45098,"num_write":215,"num_write_kb":8331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'629","reported_seq":3941,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.453295+0000","last_change":"2024-08-18T13:43:55.862627+0000","last_active":"2024-08-18T13:51:49.453295+0000","last_peered":"2024-08-18T13:51:49.453295+0000","last_clean":"2024-08-18T13:51:49.453295+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:51:49.453295+0000","last_undegraded":"2024-08-18T13:51:49.453295+0000","last_fullsized":"2024-08-18T13:51:49.453295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084356780000000003,"stat_sum":{"num_bytes":50488256,"num_objects":306,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":1250,"num_read_kb":43248,"num_write":629,"num_write_kb":49603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'84","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779153+0000","last_change":"2024-08-18T13:43:55.551442+0000","last_active":"2024-08-18T13:51:51.779153+0000","last_peered":"2024-08-18T13:51:51.779153+0000","last_clean":"2024-08-18T13:51:51.779153+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:51:51.779153+0000","last_undegraded":"2024-08-18T13:51:51.779153+0000","last_fullsized":"2024-08-18T13:51:51.779153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065977270000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"54'232","reported_seq":305,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.581418+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:51:41.581418+0000","last_peered":"2024-08-18T13:51:41.581418+0000","last_clean":"2024-08-18T13:51:41.581418+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:51:41.581418+0000","last_undegraded":"2024-08-18T13:51:41.581418+0000","last_fullsized":"2024-08-18T13:51:41.581418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583572,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":53278,"num_write":225,"num_write_kb":12431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"54'671","reported_seq":3262,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.458342+0000","last_change":"2024-08-18T13:43:56.142175+0000","last_active":"2024-08-18T13:51:49.458342+0000","last_peered":"2024-08-18T13:51:49.458342+0000","last_clean":"2024-08-18T13:51:49.458342+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:51:49.458342+0000","last_undegraded":"2024-08-18T13:51:49.458342+0000","last_fullsized":"2024-08-18T13:51:49.458342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:50.593030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076714670000000004,"stat_sum":{"num_bytes":16086174,"num_objects":343,"num_object_clones":0,"num_object_copies":1372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":529,"num_read_kb":12373,"num_write":668,"num_write_kb":24206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"54'86","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783436+0000","last_change":"2024-08-18T13:43:55.559262+0000","last_active":"2024-08-18T13:51:51.783436+0000","last_peered":"2024-08-18T13:51:51.783436+0000","last_clean":"2024-08-18T13:51:51.783436+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:51:51.783436+0000","last_undegraded":"2024-08-18T13:51:51.783436+0000","last_fullsized":"2024-08-18T13:51:51.783436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:26.470518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039212459999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"53'169","reported_seq":227,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.276349+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:50:57.276349+0000","last_peered":"2024-08-18T13:50:57.276349+0000","last_clean":"2024-08-18T13:50:57.276349+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:50:57.276349+0000","last_undegraded":"2024-08-18T13:50:57.276349+0000","last_fullsized":"2024-08-18T13:50:57.276349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8216,"num_write":168,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'644","reported_seq":3278,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.432661+0000","last_change":"2024-08-18T13:43:55.559022+0000","last_active":"2024-08-18T13:51:49.432661+0000","last_peered":"2024-08-18T13:51:49.432661+0000","last_clean":"2024-08-18T13:51:49.432661+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:51:49.432661+0000","last_undegraded":"2024-08-18T13:51:49.432661+0000","last_fullsized":"2024-08-18T13:51:49.432661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:59:17.096489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059375670000000004,"stat_sum":{"num_bytes":15740586,"num_objects":325,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":572,"num_read_kb":18359,"num_write":643,"num_write_kb":23895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"54'89","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778006+0000","last_change":"2024-08-18T13:43:56.153537+0000","last_active":"2024-08-18T13:51:51.778006+0000","last_peered":"2024-08-18T13:51:51.778006+0000","last_clean":"2024-08-18T13:51:51.778006+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:51:51.778006+0000","last_undegraded":"2024-08-18T13:51:51.778006+0000","last_fullsized":"2024-08-18T13:51:51.778006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057226309999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"54'233","reported_seq":282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.891732+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:50:56.891732+0000","last_peered":"2024-08-18T13:50:56.891732+0000","last_clean":"2024-08-18T13:50:56.891732+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:50:56.891732+0000","last_undegraded":"2024-08-18T13:50:56.891732+0000","last_fullsized":"2024-08-18T13:50:56.891732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":20397,"num_write":232,"num_write_kb":4254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"54'677","reported_seq":3169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.442883+0000","last_change":"2024-08-18T13:43:55.553346+0000","last_active":"2024-08-18T13:51:49.442883+0000","last_peered":"2024-08-18T13:51:49.442883+0000","last_clean":"2024-08-18T13:51:49.442883+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:51:49.442883+0000","last_undegraded":"2024-08-18T13:51:49.442883+0000","last_fullsized":"2024-08-18T13:51:49.442883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073620359999999998,"stat_sum":{"num_bytes":31240743,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":431,"num_read_kb":13577,"num_write":670,"num_write_kb":34133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"54'106","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778915+0000","last_change":"2024-08-18T13:43:55.861465+0000","last_active":"2024-08-18T13:51:51.778915+0000","last_peered":"2024-08-18T13:51:51.778915+0000","last_clean":"2024-08-18T13:51:51.778915+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:51:51.778915+0000","last_undegraded":"2024-08-18T13:51:51.778915+0000","last_fullsized":"2024-08-18T13:51:51.778915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059613369999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"54'9","reported_seq":51,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.581557+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:51:41.581557+0000","last_peered":"2024-08-18T13:51:41.581557+0000","last_clean":"2024-08-18T13:51:41.581557+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:51:41.581557+0000","last_undegraded":"2024-08-18T13:51:41.581557+0000","last_fullsized":"2024-08-18T13:51:41.581557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"54'699","reported_seq":3480,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.480893+0000","last_change":"2024-08-18T13:43:55.553902+0000","last_active":"2024-08-18T13:51:49.480893+0000","last_peered":"2024-08-18T13:51:49.480893+0000","last_clean":"2024-08-18T13:51:49.480893+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:51:49.480893+0000","last_undegraded":"2024-08-18T13:51:49.480893+0000","last_fullsized":"2024-08-18T13:51:49.480893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076808309999999999,"stat_sum":{"num_bytes":29636158,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":719,"num_read_kb":24704,"num_write":698,"num_write_kb":38783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"54'650","reported_seq":3292,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.034549+0000","last_change":"2024-08-18T13:43:56.152924+0000","last_active":"2024-08-18T13:51:49.034549+0000","last_peered":"2024-08-18T13:51:49.034549+0000","last_clean":"2024-08-18T13:51:49.034549+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:51:49.034549+0000","last_undegraded":"2024-08-18T13:51:49.034549+0000","last_fullsized":"2024-08-18T13:51:49.034549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068864499999999997,"stat_sum":{"num_bytes":24960867,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":580,"num_read_kb":20952,"num_write":650,"num_write_kb":24715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"54'91","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761698+0000","last_change":"2024-08-18T13:43:56.139883+0000","last_active":"2024-08-18T13:51:51.761698+0000","last_peered":"2024-08-18T13:51:51.761698+0000","last_clean":"2024-08-18T13:51:51.761698+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:51:51.761698+0000","last_undegraded":"2024-08-18T13:51:51.761698+0000","last_fullsized":"2024-08-18T13:51:51.761698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066095759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"54'199","reported_seq":252,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:42.157051+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:51:42.157051+0000","last_peered":"2024-08-18T13:51:42.157051+0000","last_clean":"2024-08-18T13:51:42.157051+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:51:42.157051+0000","last_undegraded":"2024-08-18T13:51:42.157051+0000","last_fullsized":"2024-08-18T13:51:42.157051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":197,"num_write_kb":12441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"54'658","reported_seq":3187,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.395492+0000","last_change":"2024-08-18T13:43:55.958984+0000","last_active":"2024-08-18T13:51:49.395492+0000","last_peered":"2024-08-18T13:51:49.395492+0000","last_clean":"2024-08-18T13:51:49.395492+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:51:49.395492+0000","last_undegraded":"2024-08-18T13:51:49.395492+0000","last_fullsized":"2024-08-18T13:51:49.395492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047613839999999996,"stat_sum":{"num_bytes":20867535,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":467,"num_read_kb":16390,"num_write":656,"num_write_kb":20726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764935+0000","last_change":"2024-08-18T13:43:55.552907+0000","last_active":"2024-08-18T13:51:51.764935+0000","last_peered":"2024-08-18T13:51:51.764935+0000","last_clean":"2024-08-18T13:51:51.764935+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:51:51.764935+0000","last_undegraded":"2024-08-18T13:51:51.764935+0000","last_fullsized":"2024-08-18T13:51:51.764935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068339799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"53'103","reported_seq":150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.907236+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:50:51.907236+0000","last_peered":"2024-08-18T13:50:51.907236+0000","last_clean":"2024-08-18T13:50:51.907236+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:50:51.907236+0000","last_undegraded":"2024-08-18T13:50:51.907236+0000","last_fullsized":"2024-08-18T13:50:51.907236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":103,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"54'131","reported_seq":186,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:34.672086+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:51:34.672086+0000","last_peered":"2024-08-18T13:51:34.672086+0000","last_clean":"2024-08-18T13:51:34.672086+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:51:34.672086+0000","last_undegraded":"2024-08-18T13:51:34.672086+0000","last_fullsized":"2024-08-18T13:51:34.672086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":128,"num_write_kb":12417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"54'91","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.768070+0000","last_change":"2024-08-18T13:43:56.154716+0000","last_active":"2024-08-18T13:51:51.768070+0000","last_peered":"2024-08-18T13:51:51.768070+0000","last_clean":"2024-08-18T13:51:51.768070+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:51:51.768070+0000","last_undegraded":"2024-08-18T13:51:51.768070+0000","last_fullsized":"2024-08-18T13:51:51.768070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:47:05.420697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053680339999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"54'664","reported_seq":3634,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.568805+0000","last_change":"2024-08-18T13:43:55.917879+0000","last_active":"2024-08-18T13:51:49.568805+0000","last_peered":"2024-08-18T13:51:49.568805+0000","last_clean":"2024-08-18T13:51:49.568805+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:51:49.568805+0000","last_undegraded":"2024-08-18T13:51:49.568805+0000","last_fullsized":"2024-08-18T13:51:49.568805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035122529999999999,"stat_sum":{"num_bytes":30328916,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":908,"num_read_kb":29115,"num_write":663,"num_write_kb":34071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"54'14","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.993056+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:51:38.993056+0000","last_peered":"2024-08-18T13:51:38.993056+0000","last_clean":"2024-08-18T13:51:38.993056+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:51:38.993056+0000","last_undegraded":"2024-08-18T13:51:38.993056+0000","last_fullsized":"2024-08-18T13:51:38.993056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":19,"num_write_kb":4249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757533+0000","last_change":"2024-08-18T13:43:55.555770+0000","last_active":"2024-08-18T13:51:51.757533+0000","last_peered":"2024-08-18T13:51:51.757533+0000","last_clean":"2024-08-18T13:51:51.757533+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:51:51.757533+0000","last_undegraded":"2024-08-18T13:51:51.757533+0000","last_fullsized":"2024-08-18T13:51:51.757533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:03.641291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072935329999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"54'656","reported_seq":3234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.680798+0000","last_change":"2024-08-18T13:43:55.555171+0000","last_active":"2024-08-18T13:51:49.680798+0000","last_peered":"2024-08-18T13:51:49.680798+0000","last_clean":"2024-08-18T13:51:49.680798+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:51:49.680798+0000","last_undegraded":"2024-08-18T13:51:49.680798+0000","last_fullsized":"2024-08-18T13:51:49.680798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063112100000000003,"stat_sum":{"num_bytes":21201155,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":516,"num_read_kb":24278,"num_write":657,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"54'107","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764519+0000","last_change":"2024-08-18T13:43:55.959563+0000","last_active":"2024-08-18T13:51:51.764519+0000","last_peered":"2024-08-18T13:51:51.764519+0000","last_clean":"2024-08-18T13:51:51.764519+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:51:51.764519+0000","last_undegraded":"2024-08-18T13:51:51.764519+0000","last_fullsized":"2024-08-18T13:51:51.764519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045422170000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"54'110","reported_seq":2173,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776910+0000","last_change":"2024-08-18T13:43:55.919976+0000","last_active":"2024-08-18T13:51:51.776910+0000","last_peered":"2024-08-18T13:51:51.776910+0000","last_clean":"2024-08-18T13:51:51.776910+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:51:51.776910+0000","last_undegraded":"2024-08-18T13:51:51.776910+0000","last_fullsized":"2024-08-18T13:51:51.776910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032950929999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"54'101","reported_seq":2174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757889+0000","last_change":"2024-08-18T13:43:55.863304+0000","last_active":"2024-08-18T13:51:51.757889+0000","last_peered":"2024-08-18T13:51:51.757889+0000","last_clean":"2024-08-18T13:51:51.757889+0000","last_became_active":"2024-08-18T13:40:16.610568+0000","last_became_peered":"2024-08-18T13:40:16.610568+0000","last_unstale":"2024-08-18T13:51:51.757889+0000","last_undegraded":"2024-08-18T13:51:51.757889+0000","last_fullsized":"2024-08-18T13:51:51.757889+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:53:53.687401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066503589999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"54'99","reported_seq":2163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766416+0000","last_change":"2024-08-18T13:43:56.153594+0000","last_active":"2024-08-18T13:51:51.766416+0000","last_peered":"2024-08-18T13:51:51.766416+0000","last_clean":"2024-08-18T13:51:51.766416+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:51:51.766416+0000","last_undegraded":"2024-08-18T13:51:51.766416+0000","last_fullsized":"2024-08-18T13:51:51.766416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056450420000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"54'80","reported_seq":2143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764106+0000","last_change":"2024-08-18T13:43:55.554865+0000","last_active":"2024-08-18T13:51:51.764106+0000","last_peered":"2024-08-18T13:51:51.764106+0000","last_clean":"2024-08-18T13:51:51.764106+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:51:51.764106+0000","last_undegraded":"2024-08-18T13:51:51.764106+0000","last_fullsized":"2024-08-18T13:51:51.764106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068042459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"54'74","reported_seq":2138,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761397+0000","last_change":"2024-08-18T13:43:56.142479+0000","last_active":"2024-08-18T13:51:51.761397+0000","last_peered":"2024-08-18T13:51:51.761397+0000","last_clean":"2024-08-18T13:51:51.761397+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:51:51.761397+0000","last_undegraded":"2024-08-18T13:51:51.761397+0000","last_fullsized":"2024-08-18T13:51:51.761397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079157710000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779808+0000","last_change":"2024-08-18T13:43:55.959839+0000","last_active":"2024-08-18T13:51:51.779808+0000","last_peered":"2024-08-18T13:51:51.779808+0000","last_clean":"2024-08-18T13:51:51.779808+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:51:51.779808+0000","last_undegraded":"2024-08-18T13:51:51.779808+0000","last_fullsized":"2024-08-18T13:51:51.779808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029546540000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"54'85","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755227+0000","last_change":"2024-08-18T13:43:55.555415+0000","last_active":"2024-08-18T13:51:51.755227+0000","last_peered":"2024-08-18T13:51:51.755227+0000","last_clean":"2024-08-18T13:51:51.755227+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:51:51.755227+0000","last_undegraded":"2024-08-18T13:51:51.755227+0000","last_fullsized":"2024-08-18T13:51:51.755227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055234810000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"54'92","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762023+0000","last_change":"2024-08-18T13:43:55.916616+0000","last_active":"2024-08-18T13:51:51.762023+0000","last_peered":"2024-08-18T13:51:51.762023+0000","last_clean":"2024-08-18T13:51:51.762023+0000","last_became_active":"2024-08-18T13:40:16.614889+0000","last_became_peered":"2024-08-18T13:40:16.614889+0000","last_unstale":"2024-08-18T13:51:51.762023+0000","last_undegraded":"2024-08-18T13:51:51.762023+0000","last_fullsized":"2024-08-18T13:51:51.762023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:37:46.326075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033397829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761272+0000","last_change":"2024-08-18T13:43:55.554199+0000","last_active":"2024-08-18T13:51:51.761272+0000","last_peered":"2024-08-18T13:51:51.761272+0000","last_clean":"2024-08-18T13:51:51.761272+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:51:51.761272+0000","last_undegraded":"2024-08-18T13:51:51.761272+0000","last_fullsized":"2024-08-18T13:51:51.761272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070000389999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"54'95","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.780212+0000","last_change":"2024-08-18T13:43:55.863255+0000","last_active":"2024-08-18T13:51:51.780212+0000","last_peered":"2024-08-18T13:51:51.780212+0000","last_clean":"2024-08-18T13:51:51.780212+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:51:51.780212+0000","last_undegraded":"2024-08-18T13:51:51.780212+0000","last_fullsized":"2024-08-18T13:51:51.780212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070451519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"54'88","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778638+0000","last_change":"2024-08-18T13:43:56.152879+0000","last_active":"2024-08-18T13:51:51.778638+0000","last_peered":"2024-08-18T13:51:51.778638+0000","last_clean":"2024-08-18T13:51:51.778638+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:51:51.778638+0000","last_undegraded":"2024-08-18T13:51:51.778638+0000","last_fullsized":"2024-08-18T13:51:51.778638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070254549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.759908+0000","last_change":"2024-08-18T13:43:55.956991+0000","last_active":"2024-08-18T13:51:51.759908+0000","last_peered":"2024-08-18T13:51:51.759908+0000","last_clean":"2024-08-18T13:51:51.759908+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:51:51.759908+0000","last_undegraded":"2024-08-18T13:51:51.759908+0000","last_fullsized":"2024-08-18T13:51:51.759908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003261548,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"54'85","reported_seq":2143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758255+0000","last_change":"2024-08-18T13:43:56.141939+0000","last_active":"2024-08-18T13:51:51.758255+0000","last_peered":"2024-08-18T13:51:51.758255+0000","last_clean":"2024-08-18T13:51:51.758255+0000","last_became_active":"2024-08-18T13:40:16.612760+0000","last_became_peered":"2024-08-18T13:40:16.612760+0000","last_unstale":"2024-08-18T13:51:51.758255+0000","last_undegraded":"2024-08-18T13:51:51.758255+0000","last_fullsized":"2024-08-18T13:51:51.758255+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:58:42.688761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076223990000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767420+0000","last_change":"2024-08-18T13:43:55.917355+0000","last_active":"2024-08-18T13:51:51.767420+0000","last_peered":"2024-08-18T13:51:51.767420+0000","last_clean":"2024-08-18T13:51:51.767420+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:51:51.767420+0000","last_undegraded":"2024-08-18T13:51:51.767420+0000","last_fullsized":"2024-08-18T13:51:51.767420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027557649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"54'94","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779547+0000","last_change":"2024-08-18T13:43:55.860909+0000","last_active":"2024-08-18T13:51:51.779547+0000","last_peered":"2024-08-18T13:51:51.779547+0000","last_clean":"2024-08-18T13:51:51.779547+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:51:51.779547+0000","last_undegraded":"2024-08-18T13:51:51.779547+0000","last_fullsized":"2024-08-18T13:51:51.779547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069473399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"54'96","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775325+0000","last_change":"2024-08-18T13:43:55.860127+0000","last_active":"2024-08-18T13:51:51.775325+0000","last_peered":"2024-08-18T13:51:51.775325+0000","last_clean":"2024-08-18T13:51:51.775325+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:51:51.775325+0000","last_undegraded":"2024-08-18T13:51:51.775325+0000","last_fullsized":"2024-08-18T13:51:51.775325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070196240000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761102+0000","last_change":"2024-08-18T13:43:56.136762+0000","last_active":"2024-08-18T13:51:51.761102+0000","last_peered":"2024-08-18T13:51:51.761102+0000","last_clean":"2024-08-18T13:51:51.761102+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:51:51.761102+0000","last_undegraded":"2024-08-18T13:51:51.761102+0000","last_fullsized":"2024-08-18T13:51:51.761102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004509931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.760407+0000","last_change":"2024-08-18T13:43:56.154301+0000","last_active":"2024-08-18T13:51:51.760407+0000","last_peered":"2024-08-18T13:51:51.760407+0000","last_clean":"2024-08-18T13:51:51.760407+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:51:51.760407+0000","last_undegraded":"2024-08-18T13:51:51.760407+0000","last_fullsized":"2024-08-18T13:51:51.760407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062343609999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"54'81","reported_seq":2144,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767251+0000","last_change":"2024-08-18T13:43:55.554507+0000","last_active":"2024-08-18T13:51:51.767251+0000","last_peered":"2024-08-18T13:51:51.767251+0000","last_clean":"2024-08-18T13:51:51.767251+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:51:51.767251+0000","last_undegraded":"2024-08-18T13:51:51.767251+0000","last_fullsized":"2024-08-18T13:51:51.767251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072121939999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"54'77","reported_seq":2141,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767294+0000","last_change":"2024-08-18T13:43:55.558437+0000","last_active":"2024-08-18T13:51:51.767294+0000","last_peered":"2024-08-18T13:51:51.767294+0000","last_clean":"2024-08-18T13:51:51.767294+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:51:51.767294+0000","last_undegraded":"2024-08-18T13:51:51.767294+0000","last_fullsized":"2024-08-18T13:51:51.767294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:25.248074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057680220000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776234+0000","last_change":"2024-08-18T13:43:55.958601+0000","last_active":"2024-08-18T13:51:51.776234+0000","last_peered":"2024-08-18T13:51:51.776234+0000","last_clean":"2024-08-18T13:51:51.776234+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:51:51.776234+0000","last_undegraded":"2024-08-18T13:51:51.776234+0000","last_fullsized":"2024-08-18T13:51:51.776234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049025989999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779265+0000","last_change":"2024-08-18T13:43:55.959710+0000","last_active":"2024-08-18T13:51:51.779265+0000","last_peered":"2024-08-18T13:51:51.779265+0000","last_clean":"2024-08-18T13:51:51.779265+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:51:51.779265+0000","last_undegraded":"2024-08-18T13:51:51.779265+0000","last_fullsized":"2024-08-18T13:51:51.779265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029569129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"54'83","reported_seq":2146,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781497+0000","last_change":"2024-08-18T13:43:55.552920+0000","last_active":"2024-08-18T13:51:51.781497+0000","last_peered":"2024-08-18T13:51:51.781497+0000","last_clean":"2024-08-18T13:51:51.781497+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:51:51.781497+0000","last_undegraded":"2024-08-18T13:51:51.781497+0000","last_fullsized":"2024-08-18T13:51:51.781497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062586680000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"54'74","reported_seq":2137,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777962+0000","last_change":"2024-08-18T13:43:55.551745+0000","last_active":"2024-08-18T13:51:51.777962+0000","last_peered":"2024-08-18T13:51:51.777962+0000","last_clean":"2024-08-18T13:51:51.777962+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:51:51.777962+0000","last_undegraded":"2024-08-18T13:51:51.777962+0000","last_fullsized":"2024-08-18T13:51:51.777962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064056340000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"54'105","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774468+0000","last_change":"2024-08-18T13:43:55.554037+0000","last_active":"2024-08-18T13:51:51.774468+0000","last_peered":"2024-08-18T13:51:51.774468+0000","last_clean":"2024-08-18T13:51:51.774468+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:51:51.774468+0000","last_undegraded":"2024-08-18T13:51:51.774468+0000","last_fullsized":"2024-08-18T13:51:51.774468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050911749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"54'93","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774885+0000","last_change":"2024-08-18T13:43:55.862998+0000","last_active":"2024-08-18T13:51:51.774885+0000","last_peered":"2024-08-18T13:51:51.774885+0000","last_clean":"2024-08-18T13:51:51.774885+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:51:51.774885+0000","last_undegraded":"2024-08-18T13:51:51.774885+0000","last_fullsized":"2024-08-18T13:51:51.774885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068654359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"54'102","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776207+0000","last_change":"2024-08-18T13:43:55.550186+0000","last_active":"2024-08-18T13:51:51.776207+0000","last_peered":"2024-08-18T13:51:51.776207+0000","last_clean":"2024-08-18T13:51:51.776207+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:51:51.776207+0000","last_undegraded":"2024-08-18T13:51:51.776207+0000","last_fullsized":"2024-08-18T13:51:51.776207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055002549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"54'76","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774945+0000","last_change":"2024-08-18T13:43:55.859954+0000","last_active":"2024-08-18T13:51:51.774945+0000","last_peered":"2024-08-18T13:51:51.774945+0000","last_clean":"2024-08-18T13:51:51.774945+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:51:51.774945+0000","last_undegraded":"2024-08-18T13:51:51.774945+0000","last_fullsized":"2024-08-18T13:51:51.774945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069338170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762078+0000","last_change":"2024-08-18T13:43:56.150131+0000","last_active":"2024-08-18T13:51:51.762078+0000","last_peered":"2024-08-18T13:51:51.762078+0000","last_clean":"2024-08-18T13:51:51.762078+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:51:51.762078+0000","last_undegraded":"2024-08-18T13:51:51.762078+0000","last_fullsized":"2024-08-18T13:51:51.762078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044622389999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"54'76","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763798+0000","last_change":"2024-08-18T13:43:56.136824+0000","last_active":"2024-08-18T13:51:51.763798+0000","last_peered":"2024-08-18T13:51:51.763798+0000","last_clean":"2024-08-18T13:51:51.763798+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:51:51.763798+0000","last_undegraded":"2024-08-18T13:51:51.763798+0000","last_fullsized":"2024-08-18T13:51:51.763798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035965799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"54'86","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778822+0000","last_change":"2024-08-18T13:43:55.552484+0000","last_active":"2024-08-18T13:51:51.778822+0000","last_peered":"2024-08-18T13:51:51.778822+0000","last_clean":"2024-08-18T13:51:51.778822+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:51:51.778822+0000","last_undegraded":"2024-08-18T13:51:51.778822+0000","last_fullsized":"2024-08-18T13:51:51.778822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067690759999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":938443088,"num_objects":10906,"num_object_clones":0,"num_object_copies":43624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10906,"num_whiteouts":0,"num_read":21895,"num_read_kb":705026,"num_write":21484,"num_write_kb":1061752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2001649664,"data_stored":2001649664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5769,"ondisk_log_size":5769,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":184495130,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":586,"num_read_kb":685208,"num_write":6404,"num_write_kb":185940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":369049600,"data_stored":368990260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6329,"ondisk_log_size":6329,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599760,"kb_used_data":294088,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93772080,"statfs":{"total":96636764160,"available":96022609920,"internally_reserved":0,"allocated":301146112,"data_stored":300847166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2999999999999998,"5min":1.026,"15min":0.878},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":25.744,"5min":25.744,"15min":25.744},"last":1.254},{"interface":"front","average":{"1min":1.3149999999999999,"5min":0.84999999999999998,"15min":0.82599999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.33100000000000002},"max":{"1min":8.1679999999999993,"5min":8.1679999999999993,"15min":8.1679999999999993},"last":0.93799999999999994}]},{"osd":1,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":0.90700000000000003,"15min":0.98599999999999999},"min":{"1min":0.5,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":10.151999999999999,"5min":10.151999999999999,"15min":39.203000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":2.323,"5min":1.111,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":25.852,"5min":25.852,"15min":25.852},"last":0.96499999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":1.121,"15min":0.95099999999999996},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.39400000000000002},"max":{"1min":25.818999999999999,"5min":25.818999999999999,"15min":25.818999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.85199999999999998},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.35799999999999998},"max":{"1min":1.133,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.0840000000000001,"15min":1.135},"min":{"1min":0.64700000000000002,"5min":0.42699999999999999,"15min":0.377},"max":{"1min":17.300999999999998,"5min":17.300999999999998,"15min":42.905000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":2.4689999999999999,"5min":1.238,"15min":1.115},"min":{"1min":0.75600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":25.786999999999999,"5min":25.786999999999999,"15min":25.786999999999999},"last":1.323}]},{"osd":4,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.29699999999999999},"max":{"1min":8.3870000000000005,"5min":8.3870000000000005,"15min":8.3870000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.171,"15min":1.23},"min":{"1min":0.79800000000000004,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":17.056999999999999,"5min":17.056999999999999,"15min":52.673000000000002},"last":1.127}]},{"osd":5,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":1.274,"15min":1.119},"min":{"1min":0.44,"5min":0.44,"15min":0.33700000000000002},"max":{"1min":25.881,"5min":25.881,"15min":25.881},"last":1.087},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.1080000000000001,"15min":1.073},"min":{"1min":0.73899999999999999,"5min":0.54200000000000004,"15min":0.37},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":0.77100000000000002}]},{"osd":6,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.1659999999999999,"15min":1.1259999999999999},"min":{"1min":0.83799999999999997,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":8.0510000000000002,"5min":8.0510000000000002,"15min":8.0510000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.1990000000000001,"15min":1.26},"min":{"1min":0.85499999999999998,"5min":0.49399999999999999,"15min":0.435},"max":{"1min":10.523,"5min":10.523,"15min":39.161000000000001},"last":1.02}]}]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610072,"kb_used_data":312720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93761768,"statfs":{"total":96636764160,"available":96012050432,"internally_reserved":0,"allocated":320225280,"data_stored":319926507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76500000000000001,"15min":0.88},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.165,"5min":1.21,"15min":4.532},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.248},"max":{"1min":1.258,"5min":1.3200000000000001,"15min":1.9650000000000001},"last":0.73899999999999999}]},{"osd":1,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.92000000000000004},"min":{"1min":0.42399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2749999999999999,"5min":1.294,"15min":4.468},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.91900000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.30499999999999999},"max":{"1min":1.1990000000000001,"5min":1.286,"15min":2.3620000000000001},"last":1.01}]},{"osd":2,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77700000000000002,"15min":0.83899999999999997},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.28000000000000003},"max":{"1min":1.085,"5min":1.177,"15min":1.413},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.97299999999999998},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1060000000000001,"5min":1.468,"15min":4.5},"last":0.69999999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88700000000000001,"15min":0.94199999999999995},"min":{"1min":0.60899999999999999,"5min":0.41999999999999998,"15min":0.35399999999999998},"max":{"1min":1.1919999999999999,"5min":1.276,"15min":4.3250000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.91600000000000004},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.32900000000000001},"max":{"1min":1.0920000000000001,"5min":1.306,"15min":1.8069999999999999},"last":0.94899999999999995}]},{"osd":4,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92700000000000005,"15min":1.0109999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.228,"5min":1.552,"15min":5.2549999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.7230000000000001,"15min":4.6600000000000001},"last":1.145}]},{"osd":5,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":1.0129999999999999},"min":{"1min":0.54900000000000004,"5min":0.45400000000000001,"15min":0.373},"max":{"1min":1.179,"5min":1.3160000000000001,"15min":4.4009999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.042,"15min":1.1499999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.272,"5min":1.6879999999999999,"15min":6.1260000000000003},"last":1.125}]},{"osd":7,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0669999999999999,"15min":1.111},"min":{"1min":0.67600000000000005,"5min":0.55500000000000005,"15min":0.40500000000000003},"max":{"1min":1.3009999999999999,"5min":1.4830000000000001,"15min":1.9299999999999999},"last":1.103},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0609999999999999,"15min":1.1080000000000001},"min":{"1min":0.67500000000000004,"5min":0.46300000000000002,"15min":0.40200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":2.1150000000000002},"last":1.2330000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509594,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570040,"kb_used_data":272688,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93801800,"statfs":{"total":96636764160,"available":96053043200,"internally_reserved":0,"allocated":279232512,"data_stored":278928482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.24299999999999999},"max":{"1min":1.1679999999999999,"5min":1.204,"15min":1.22},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":1.085,"5min":1.339,"15min":1.73},"last":0.84399999999999997}]},{"osd":1,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82499999999999996,"15min":0.86799999999999999},"min":{"1min":0.54100000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.2909999999999999,"5min":1.415,"15min":3.6299999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.36699999999999999,"15min":0.27600000000000002},"max":{"1min":1.0589999999999999,"5min":1.28,"15min":1.5169999999999999},"last":1.139}]},{"osd":2,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.85799999999999998},"min":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.5920000000000001},"last":0.77000000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92100000000000004,"15min":0.91800000000000004},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.309},"max":{"1min":1.111,"5min":1.387,"15min":1.45},"last":1.222},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.641,"15min":1.641},"last":1.111}]},{"osd":5,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.313},"max":{"1min":1.3640000000000001,"5min":1.575,"15min":3.7789999999999999},"last":1.081},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.998,"15min":1.0149999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3129999999999999,"5min":1.754,"15min":1.754},"last":0.99399999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0580000000000001,"15min":1.079},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.504},"max":{"1min":1.6259999999999999,"5min":1.901,"15min":7.8490000000000002},"last":1.026},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.016,"15min":1.0629999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2809999999999999,"5min":1.5229999999999999,"15min":6.3380000000000001},"last":1.282}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40000000000000002},"max":{"1min":1.2470000000000001,"5min":1.7470000000000001,"15min":1.8400000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.151,"15min":1.1819999999999999},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.42399999999999999},"max":{"1min":1.6850000000000001,"5min":2.4569999999999999,"15min":3.7410000000000001},"last":1.165}]}]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683436,"kb_used_data":373604,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93688404,"statfs":{"total":96636764160,"available":95936925696,"internally_reserved":0,"allocated":382570496,"data_stored":382269906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.29499999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.89500000000000002,"5min":1.2050000000000001,"15min":1.2170000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67800000000000005,"15min":0.69299999999999995},"min":{"1min":0.44900000000000001,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":1.413,"15min":1.413},"last":0.91400000000000003}]},{"osd":1,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.68600000000000005},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86499999999999999,"5min":1.46,"15min":3.4820000000000002},"last":1.089},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.751},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.8979999999999999},"last":0.96199999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.48599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1910000000000001,"5min":1.6419999999999999,"15min":2.415},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.87,"15min":0.86899999999999999},"min":{"1min":0.57799999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.2589999999999999,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.0580000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.173,"5min":1.681,"15min":3.2490000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.052},"min":{"1min":0.66300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5189999999999999,"5min":2.4870000000000001,"15min":3.8799999999999999},"last":1.2270000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.996,"15min":4.1059999999999999},"last":1.119},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0089999999999999,"15min":1.05},"min":{"1min":0.63200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1719999999999999,"5min":1.627,"15min":3.2799999999999998},"last":1.1459999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.504,"5min":0.504,"15min":0.41999999999999998},"max":{"1min":1.3480000000000001,"5min":1.833,"15min":2.1080000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.054,"15min":1.0880000000000001},"min":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.573,"15min":2.7469999999999999},"last":1.3120000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4419999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.169},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.087,"15min":1.0960000000000001},"min":{"1min":0.71499999999999997,"5min":0.63200000000000001,"15min":0.51100000000000001},"max":{"1min":1.583,"5min":1.9039999999999999,"15min":2.1429999999999998},"last":1.337}]}]},{"osd":2,"up_from":14,"seq":60129542298,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530780,"kb_used_data":229268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93841060,"statfs":{"total":96636764160,"available":96093245440,"internally_reserved":0,"allocated":234770432,"data_stored":234477374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.186},"max":{"1min":1.0369999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":1.0960000000000001,"5min":3.27,"15min":3.27},"last":0.5}]},{"osd":1,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.038,"5min":1.1040000000000001,"15min":1.696},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":3.2290000000000001,"15min":3.2290000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.78200000000000003},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.253},"max":{"1min":0.995,"5min":1.077,"15min":12.468999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.84399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35899999999999999,"15min":0.27500000000000002},"max":{"1min":1.077,"5min":1.3540000000000001,"15min":1.744},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2709999999999999,"5min":1.681,"15min":1.8480000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.94799999999999995},"min":{"1min":0.46899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2609999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1759999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39000000000000001},"max":{"1min":1.149,"5min":1.498,"15min":1.627},"last":0.96199999999999997}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96899999999999997,"15min":0.98799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.42199999999999999},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.649},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.0309999999999999,"15min":1.0649999999999999},"min":{"1min":0.67000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":1.5820000000000001,"15min":1.905},"last":0.878}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0549999999999999,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.60399999999999998,"15min":0.436},"max":{"1min":1.3140000000000001,"5min":1.5089999999999999,"15min":1.9490000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.0149999999999999},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.46400000000000002},"max":{"1min":1.343,"5min":1.4379999999999999,"15min":1.7669999999999999},"last":0.90200000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542298,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677976,"kb_used_data":376464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93693864,"statfs":{"total":96636764160,"available":95942516736,"internally_reserved":0,"allocated":385499136,"data_stored":385199863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.65200000000000002},"min":{"1min":0.28899999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.94899999999999995,"5min":1.3009999999999999,"15min":3.1160000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.72099999999999997},"min":{"1min":0.30599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.107,"5min":1.327,"15min":3.718},"last":0.76900000000000002}]},{"osd":2,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0549999999999999,"5min":4.968,"15min":4.968},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33400000000000002,"15min":0.20899999999999999},"max":{"1min":1.327,"5min":1.3560000000000001,"15min":9.3309999999999995},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4810000000000001,"15min":1.927},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79400000000000004,"15min":0.84899999999999998},"min":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":1.46,"5min":1.46,"15min":6.8559999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.28000000000000003},"max":{"1min":1.488,"5min":5.1020000000000003,"15min":5.6529999999999996},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.63400000000000001,"5min":0.39700000000000002,"15min":0.28599999999999998},"max":{"1min":1.4410000000000001,"5min":1.554,"15min":1.5740000000000001},"last":1.125}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3859999999999999,"5min":1.546,"15min":3.7570000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1,"5min":1.04,"15min":1.1040000000000001},"min":{"1min":0.47799999999999998,"5min":0.46400000000000002,"15min":0.45700000000000002},"max":{"1min":1.591,"5min":5.1799999999999997,"15min":19.027999999999999},"last":1.272}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.40699999999999997},"max":{"1min":1.5529999999999999,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.113,"5min":1.0880000000000001,"15min":1.131},"min":{"1min":0.54900000000000004,"5min":0.49099999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.861},"last":0.98799999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.069,"15min":1.119},"min":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58799999999999997},"max":{"1min":1.383,"5min":1.526,"15min":3.8100000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40000000000000002},"max":{"1min":1.431,"5min":5.3099999999999996,"15min":5.3099999999999996},"last":1.3220000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542298,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549780,"kb_used_data":248268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93822060,"statfs":{"total":96636764160,"available":96073789440,"internally_reserved":0,"allocated":254226432,"data_stored":253919977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.65400000000000003},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":5.3159999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.33500000000000002,"5min":0.254,"15min":0.19500000000000001},"max":{"1min":0.93600000000000005,"5min":1.2370000000000001,"15min":3.4809999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.47099999999999997,"5min":0.32600000000000001,"15min":0.26300000000000001},"max":{"1min":0.99199999999999999,"5min":1.222,"15min":3.3029999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.28},"last":0.90000000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.288,"5min":1.288,"15min":5.399},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86799999999999999,"15min":0.93100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.33900000000000002},"max":{"1min":1.1799999999999999,"5min":1.365,"15min":5.0359999999999996},"last":0.99099999999999999}]},{"osd":4,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.37},"max":{"1min":1.3899999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":8.7799999999999994},"last":0.93999999999999995}]},{"osd":5,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96099999999999997,"15min":0.97499999999999998},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.377},{"interface":"front","average":{"1min":1.087,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.40300000000000002},"max":{"1min":1.51,"5min":1.52,"15min":1.8879999999999999},"last":0.82499999999999996}]},{"osd":6,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.77800000000000002,"5min":0.46500000000000002,"15min":0.374},"max":{"1min":1.423,"5min":1.599,"15min":3.7200000000000002},"last":1.171},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.042,"15min":1.085},"min":{"1min":0.68600000000000005,"5min":0.58099999999999996,"15min":0.52600000000000002},"max":{"1min":1.55,"5min":1.659,"15min":3.2789999999999999},"last":1.4239999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.143,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.60599999999999998,"15min":0.40300000000000002},"max":{"1min":1.514,"5min":1.704,"15min":36.295000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.171,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.55300000000000005,"15min":0.42999999999999999},"max":{"1min":1.5740000000000001,"5min":1.706,"15min":3.448},"last":1.206}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515540,"kb_used_data":214028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93856300,"statfs":{"total":96636764160,"available":96108851200,"internally_reserved":0,"allocated":219164672,"data_stored":218869425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.81100000000000005},"min":{"1min":0.48199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.135,"5min":1.2789999999999999,"15min":1.901},"last":0.747},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.81000000000000005,"15min":0.85599999999999998},"min":{"1min":0.63900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":4.0810000000000004},"last":0.88100000000000001}]},{"osd":1,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.82099999999999995},"min":{"1min":0.51900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.101,"5min":1.429,"15min":2.4809999999999999},"last":1.022},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1799999999999999,"5min":1.456,"15min":1.7749999999999999},"last":0.93400000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.63200000000000001,"5min":0.46999999999999997,"15min":0.41799999999999998},"max":{"1min":1.1419999999999999,"5min":1.524,"15min":1.774},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.874,"15min":0.93899999999999995},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.32500000000000001},"max":{"1min":1.1299999999999999,"5min":1.3160000000000001,"15min":4.1559999999999997},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.94399999999999995,"15min":0.97999999999999998},"min":{"1min":0.59799999999999998,"5min":0.54600000000000004,"15min":0.36199999999999999},"max":{"1min":1.3300000000000001,"5min":1.4410000000000001,"15min":4.5510000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.92700000000000005,"15min":0.98499999999999999},"min":{"1min":0.72199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.423,"5min":1.4470000000000001,"15min":3.976},"last":1.143}]},{"osd":4,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.7909999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0660000000000001,"15min":1.1120000000000001},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.6839999999999999,"15min":2.532},"last":1.2430000000000001}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.03,"15min":1.085},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3109999999999999,"5min":1.532,"15min":4.2050000000000001},"last":1.071},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0569999999999999,"15min":1.113},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":2.911},"last":1.2150000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.089,"15min":1.165},"min":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2989999999999999,"5min":1.659,"15min":2.569},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.1359999999999999,"15min":1.361},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40400000000000003},"max":{"1min":1.407,"5min":1.605,"15min":58.290999999999997},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41955328,"data_stored":41943666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50331808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21549056,"data_stored":21547239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34140160,"data_stored":34130811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53907456,"data_stored":53894667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41947136,"data_stored":41943130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50332308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":74870784,"data_stored":74866631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211058688,"data_stored":211058688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334540800,"data_stored":334540800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212602880,"data_stored":212602880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347811840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":224706560,"data_stored":224706560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176599040,"data_stored":176599040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":269266944,"data_stored":269266944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":225062912,"data_stored":225062912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:51:58.611 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:51:58.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.714+0000 7f47dffff700 1 Processor -- start 2024-08-18T13:51:58.720 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.714+0000 7f47dffff700 1 -- start start 2024-08-18T13:51:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.714+0000 7f47dffff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f47e014b940 0x7f47e006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.714+0000 7f47dffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e006ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e006de00 con 0x7f47e014bd10 2024-08-18T13:51:58.721 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e006fac0 con 0x7f47e014b940 2024-08-18T13:51:58.722 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0151870 con 0x7f47e00be5a0 2024-08-18T13:51:58.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47de7fc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e006ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47deffd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f47e014b940 0x7f47e006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47df7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f47e014bd10 0x7f47e00c7fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:55162/0 (socket says 172.21.15.118:55162) 2024-08-18T13:51:58.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47df7fe700 1 -- 172.21.15.118:0/560108658 learned_addr learned my addr 172.21.15.118:0/560108658 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:51:58.724 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47ddffb700 1 -- 172.21.15.118:0/560108658 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28584376 0 0) 0x7f47e006de00 con 0x7f47e014bd10 2024-08-18T13:51:58.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47de7fc700 1 -- 172.21.15.118:0/560108658 >> v2:172.21.15.178:3300/0 conn(0x7f47e014b940 msgr2=0x7f47e006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:51:58.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47de7fc700 1 --2- 172.21.15.118:0/560108658 >> v2:172.21.15.178:3300/0 conn(0x7f47e014b940 0x7f47e006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47de7fc700 1 -- 172.21.15.118:0/560108658 >> v1:172.21.15.178:6789/0 conn(0x7f47e014bd10 legacy=0x7f47e00c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47de7fc700 1 -- 172.21.15.118:0/560108658 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00c2da0 con 0x7f47e00be5a0 2024-08-18T13:51:58.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47de7fc700 1 --2- 172.21.15.118:0/560108658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e006ea00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009e60 tx=0x7f47d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9cf39e5c08a3a45 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47ddffb700 1 -- 172.21.15.118:0/560108658 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d000e070 con 0x7f47e00be5a0 2024-08-18T13:51:58.725 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47ddffb700 1 -- 172.21.15.118:0/560108658 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d000bb60 con 0x7f47e00be5a0 2024-08-18T13:51:58.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47ddffb700 1 -- 172.21.15.118:0/560108658 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0012430 con 0x7f47e00be5a0 2024-08-18T13:51:58.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- 172.21.15.118:0/560108658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 msgr2=0x7f47e006ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 --2- 172.21.15.118:0/560108658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e006ea00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009e60 tx=0x7f47d0009780 comp rx=0 tx=0).stop 2024-08-18T13:51:58.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- 172.21.15.118:0/560108658 shutdown_connections 2024-08-18T13:51:58.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 --2- 172.21.15.118:0/560108658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e006ea00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 --2- 172.21.15.118:0/560108658 >> v2:172.21.15.178:3300/0 conn(0x7f47e014b940 0x7f47e006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- 172.21.15.118:0/560108658 >> 172.21.15.118:0/560108658 conn(0x7f47e006c020 msgr2=0x7f47e00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:58.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- 172.21.15.118:0/560108658 shutdown_connections 2024-08-18T13:51:58.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.718+0000 7f47dffff700 1 -- 172.21.15.118:0/560108658 wait complete. 2024-08-18T13:51:58.729 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 Processor -- start 2024-08-18T13:51:58.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 -- start start 2024-08-18T13:51:58.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e00c0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47e014b940 0x7f47e013f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e014bd10 0x7f47e013fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0140280 con 0x7f47e014b940 2024-08-18T13:51:58.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e014a790 con 0x7f47e014bd10 2024-08-18T13:51:58.730 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e013c060 con 0x7f47e00be5a0 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47e014b940 0x7f47e013f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47e014b940 0x7f47e013f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44960/0 (socket says 172.21.15.118:44960) 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 -- 172.21.15.118:0/4111478081 learned_addr learned my addr 172.21.15.118:0/4111478081 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47deffd700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e00c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47df7fe700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e014bd10 0x7f47e013fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 -- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 msgr2=0x7f47e00c0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e00c0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.731 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 -- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e014bd10 msgr2=0x7f47e013fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e014bd10 0x7f47e013fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 -- 172.21.15.118:0/4111478081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0009a20 con 0x7f47e014b940 2024-08-18T13:51:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47df7fe700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e014bd10 0x7f47e013fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:51:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47de7fc700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47e014b940 0x7f47e013f800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f47e00701b0 tx=0x7f47d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47c6ffd700 1 -- 172.21.15.118:0/4111478081 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0012d30 con 0x7f47e014b940 2024-08-18T13:51:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47c6ffd700 1 -- 172.21.15.118:0/4111478081 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d0027470 con 0x7f47e014b940 2024-08-18T13:51:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e013ba90 con 0x7f47e014b940 2024-08-18T13:51:58.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47c6ffd700 1 -- 172.21.15.118:0/4111478081 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0027830 con 0x7f47e014b940 2024-08-18T13:51:58.733 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.722+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e014a4b0 con 0x7f47e014b940 2024-08-18T13:51:58.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.726+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e00713a0 con 0x7f47e014b940 2024-08-18T13:51:58.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.726+0000 7f47c6ffd700 1 -- 172.21.15.118:0/4111478081 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f47d0024070 con 0x7f47e014b940 2024-08-18T13:51:58.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.726+0000 7f47c6ffd700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f47cc060860 0x7f47cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:51:58.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.726+0000 7f47c6ffd700 1 -- 172.21.15.118:0/4111478081 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f47d007f6f0 con 0x7f47e014b940 2024-08-18T13:51:58.734 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.726+0000 7f47deffd700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f47cc060860 0x7f47cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:51:58.735 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.726+0000 7f47deffd700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f47cc060860 0x7f47cc062d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009db0 tx=0x7f47d40091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:51:58.737 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.734+0000 7f47c6ffd700 1 -- 172.21.15.118:0/4111478081 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f47d004f580 con 0x7f47e014b940 2024-08-18T13:51:58.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.898+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47e013c1a0 con 0x7f47e014b940 2024-08-18T13:51:58.903 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.898+0000 7f47c6ffd700 1 -- 172.21.15.118:0/4111478081 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18405 (secure 0 0 0) 0x7f47d0052ba0 con 0x7f47e014b940 2024-08-18T13:51:58.904 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:51:58.904 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":54,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:50:50.639291+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/2054758182":"2024-08-19T13:50:50.639112+0000","172.21.15.178:6836/1054354354":"2024-08-19T13:49:30.622514+0000","172.21.15.178:6837/3452743394":"2024-08-19T13:45:35.570038+0000","172.21.15.178:6832/679522662":"2024-08-19T13:47:20.593054+0000","172.21.15.178:6836/3452743394":"2024-08-19T13:45:35.570038+0000","172.21.15.178:6832/1590569622":"2024-08-19T13:44:15.544758+0000","172.21.15.178:6832/2054758182":"2024-08-19T13:50:50.639112+0000","172.21.15.178:6837/1054354354":"2024-08-19T13:49:30.622514+0000","172.21.15.178:6835/679522662":"2024-08-19T13:47:20.593054+0000","172.21.15.178:6835/1590569622":"2024-08-19T13:44:15.544758+0000","172.21.15.178:6837/2503344632":"2024-08-19T13:42:15.510903+0000","172.21.15.178:6836/2503344632":"2024-08-19T13:42:15.510903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:51:58.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f47cc060860 msgr2=0x7f47cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f47cc060860 0x7f47cc062d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009db0 tx=0x7f47d40091a0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47e014b940 msgr2=0x7f47e013f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:51:58.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47e014b940 0x7f47e013f800 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f47e00701b0 tx=0x7f47d0019000 comp rx=0 tx=0).stop 2024-08-18T13:51:58.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 shutdown_connections 2024-08-18T13:51:58.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f47cc060860 0x7f47cc062d10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e014bd10 0x7f47e013fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47e014b940 0x7f47e013f800 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 --2- 172.21.15.118:0/4111478081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47e00be5a0 0x7f47e00c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:51:58.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 >> 172.21.15.118:0/4111478081 conn(0x7f47e006c020 msgr2=0x7f47e014fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:51:58.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 shutdown_connections 2024-08-18T13:51:58.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:51:58.902+0000 7f47dffff700 1 -- 172.21.15.118:0/4111478081 wait complete. 2024-08-18T13:51:59.924 INFO:tasks.ceph:Scrubbing osd.0 2024-08-18T13:51:59.925 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:00.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.046+0000 7f2db9594700 1 Processor -- start 2024-08-18T13:52:00.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.046+0000 7f2db9594700 1 -- start start 2024-08-18T13:52:00.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db4069b70 0x7f2db40c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.059 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2db40c9620 0x7f2db414c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db41532a0 con 0x7f2db40697a0 2024-08-18T13:52:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db4068080 con 0x7f2db40c9620 2024-08-18T13:52:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4153e40 con 0x7f2db4069b70 2024-08-18T13:52:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db3fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2db40697a0 0x7f2db4153730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38262/0 (socket says 172.21.15.118:38262) 2024-08-18T13:52:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db3fff700 1 -- 172.21.15.118:0/2256221752 learned_addr learned my addr 172.21.15.118:0/2256221752 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:00.060 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db37fe700 1 --2- 172.21.15.118:0/2256221752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db4069b70 0x7f2db40c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db2ffd700 1 -- 172.21.15.118:0/2256221752 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949057482 0 0) 0x7f2db41532a0 con 0x7f2db40697a0 2024-08-18T13:52:00.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db2ffd700 1 -- 172.21.15.118:0/2256221752 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da0003570 con 0x7f2db40697a0 2024-08-18T13:52:00.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db37fe700 1 -- 172.21.15.118:0/2256221752 >> v2:172.21.15.178:3300/0 conn(0x7f2db40c9620 msgr2=0x7f2db414c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:00.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db37fe700 1 --2- 172.21.15.118:0/2256221752 >> v2:172.21.15.178:3300/0 conn(0x7f2db40c9620 0x7f2db414c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.061 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db37fe700 1 -- 172.21.15.118:0/2256221752 >> v1:172.21.15.118:6789/0 conn(0x7f2db40697a0 legacy=0x7f2db4153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db37fe700 1 -- 172.21.15.118:0/2256221752 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4152f90 con 0x7f2db4069b70 2024-08-18T13:52:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db37fe700 1 --2- 172.21.15.118:0/2256221752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db4069b70 0x7f2db40c90e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0098e0 tx=0x7f2d9c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677d32cc7b310703 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db2ffd700 1 -- 172.21.15.118:0/2256221752 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00b640 con 0x7f2db4069b70 2024-08-18T13:52:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db2ffd700 1 -- 172.21.15.118:0/2256221752 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c00b7a0 con 0x7f2db4069b70 2024-08-18T13:52:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db2ffd700 1 -- 172.21.15.118:0/2256221752 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00bb40 con 0x7f2db4069b70 2024-08-18T13:52:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- 172.21.15.118:0/2256221752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db4069b70 msgr2=0x7f2db40c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.062 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 --2- 172.21.15.118:0/2256221752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db4069b70 0x7f2db40c90e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0098e0 tx=0x7f2d9c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- 172.21.15.118:0/2256221752 shutdown_connections 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 --2- 172.21.15.118:0/2256221752 >> v2:172.21.15.178:3300/0 conn(0x7f2db40c9620 0x7f2db414c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 --2- 172.21.15.118:0/2256221752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db4069b70 0x7f2db40c90e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- 172.21.15.118:0/2256221752 >> 172.21.15.118:0/2256221752 conn(0x7f2db406f4b0 msgr2=0x7f2db40c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- 172.21.15.118:0/2256221752 shutdown_connections 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.050+0000 7f2db9594700 1 -- 172.21.15.118:0/2256221752 wait complete. 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 Processor -- start 2024-08-18T13:52:00.063 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 -- start start 2024-08-18T13:52:00.066 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40697a0 0x7f2db4166e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db4069b70 0x7f2db41638f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2db40c9620 0x7f2db41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4162ee0 con 0x7f2db40c9620 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4163e30 con 0x7f2db4069b70 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db9594700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4163460 con 0x7f2db40697a0 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db8d93700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2db40c9620 0x7f2db41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db8d93700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2db40c9620 0x7f2db41582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44968/0 (socket says 172.21.15.118:44968) 2024-08-18T13:52:00.067 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.054+0000 7f2db8d93700 1 -- 172.21.15.118:0/2759361540 learned_addr learned my addr 172.21.15.118:0/2759361540 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db3fff700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40697a0 0x7f2db4166e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db37fe700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db4069b70 0x7f2db41638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db8d93700 1 -- 172.21.15.118:0/2759361540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40697a0 msgr2=0x7f2db4166e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db8d93700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40697a0 0x7f2db4166e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db8d93700 1 -- 172.21.15.118:0/2759361540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db4069b70 msgr2=0x7f2db41638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db8d93700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db4069b70 0x7f2db41638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db8d93700 1 -- 172.21.15.118:0/2759361540 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c009420 con 0x7f2db40c9620 2024-08-18T13:52:00.068 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db37fe700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db4069b70 0x7f2db41638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db8d93700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2db40c9620 0x7f2db41582e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b810 tx=0x7f2dac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00f510 con 0x7f2db40c9620 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac00f690 con 0x7f2db40c9620 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00fa50 con 0x7f2db40c9620 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db40c7f60 con 0x7f2db40c9620 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db40c8520 con 0x7f2db40c9620 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2db4071410 con 0x7f2db40c9620 2024-08-18T13:52:00.069 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.058+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2dac024430 con 0x7f2db40c9620 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2dabfff700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d94060890 0x7f2d94062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f2dac07d7a0 con 0x7f2db40c9620 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2db3fff700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d94060890 0x7f2d94062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2dabfff700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f2d9406a610 0x7f2d9406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 --> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d9406d170 con 0x7f2d9406a610 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dac07db70 con 0x7f2db40c9620 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2db37fe700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f2d9406a610 0x7f2d9406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2db37fe700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f2d9406a610 0x7f2d9406caa0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.070 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2db3fff700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d94060890 0x7f2d94062d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2da4005c50 tx=0x7f2da4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.071 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.062+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== osd.0 v2:172.21.15.118:6824/24281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2d9406d170 con 0x7f2d9406a610 2024-08-18T13:52:00.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 --> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2db414d2e0 con 0x7f2d9406a610 2024-08-18T13:52:00.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2dabfff700 1 -- 172.21.15.118:0/2759361540 <== osd.0 v2:172.21.15.118:6824/24281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2db414d2e0 con 0x7f2d9406a610 2024-08-18T13:52:00.084 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:00.084 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:00.084 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:00.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f2d9406a610 msgr2=0x7f2d9406caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6824/24281,v1:172.21.15.118:6825/24281] conn(0x7f2d9406a610 0x7f2d9406caa0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d94060890 msgr2=0x7f2d94062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d94060890 0x7f2d94062d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2da4005c50 tx=0x7f2da4009040 comp rx=0 tx=0).stop 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2db40c9620 msgr2=0x7f2db41582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 --2- 172.21.15.118:0/2759361540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2db40c9620 0x7f2db41582e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b810 tx=0x7f2dac0091d0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db8d93700 1 -- 172.21.15.118:0/2759361540 reap_dead start 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 shutdown_connections 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 >> 172.21.15.118:0/2759361540 conn(0x7f2db406f4b0 msgr2=0x7f2db4144f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 shutdown_connections 2024-08-18T13:52:00.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.078+0000 7f2db9594700 1 -- 172.21.15.118:0/2759361540 wait complete. 2024-08-18T13:52:00.097 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-18T13:52:00.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84c0a1a700 1 Processor -- start 2024-08-18T13:52:00.208 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84c0a1a700 1 -- start start 2024-08-18T13:52:00.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84c0a1a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84bc0be5a0 0x7f84bc151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.211 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84c0a1a700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc06de00 con 0x7f84bc14b940 2024-08-18T13:52:00.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84c0a1a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc0c2da0 con 0x7f84bc14bd10 2024-08-18T13:52:00.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84c0a1a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84bc06fac0 con 0x7f84bc0be5a0 2024-08-18T13:52:00.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84bb7fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f84bc14b940 0x7f84bc06e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38278/0 (socket says 172.21.15.118:38278) 2024-08-18T13:52:00.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.202+0000 7f84bb7fe700 1 -- 172.21.15.118:0/3409714848 learned_addr learned my addr 172.21.15.118:0/3409714848 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:00.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84baffd700 1 --2- 172.21.15.118:0/3409714848 >> v2:172.21.15.178:3300/0 conn(0x7f84bc0be5a0 0x7f84bc151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84ba7fc700 1 -- 172.21.15.118:0/3409714848 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622521913 0 0) 0x7f84bc06de00 con 0x7f84bc14b940 2024-08-18T13:52:00.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84ba7fc700 1 -- 172.21.15.118:0/3409714848 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84a4003570 con 0x7f84bc14b940 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84baffd700 1 -- 172.21.15.118:0/3409714848 >> v1:172.21.15.178:6789/0 conn(0x7f84bc14bd10 legacy=0x7f84bc15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84baffd700 1 -- 172.21.15.118:0/3409714848 >> v1:172.21.15.118:6789/0 conn(0x7f84bc14b940 legacy=0x7f84bc06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84baffd700 1 -- 172.21.15.118:0/3409714848 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc0c3080 con 0x7f84bc0be5a0 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84baffd700 1 --2- 172.21.15.118:0/3409714848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc0be5a0 0x7f84bc151cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009e60 tx=0x7f84ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49d066367a8cc191 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84ba7fc700 1 -- 172.21.15.118:0/3409714848 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00e070 con 0x7f84bc0be5a0 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84ba7fc700 1 -- 172.21.15.118:0/3409714848 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac00bb60 con 0x7f84bc0be5a0 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84ba7fc700 1 -- 172.21.15.118:0/3409714848 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac012430 con 0x7f84bc0be5a0 2024-08-18T13:52:00.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 -- 172.21.15.118:0/3409714848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc0be5a0 msgr2=0x7f84bc151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/3409714848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc0be5a0 0x7f84bc151cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009e60 tx=0x7f84ac009780 comp rx=0 tx=0).stop 2024-08-18T13:52:00.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 -- 172.21.15.118:0/3409714848 shutdown_connections 2024-08-18T13:52:00.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/3409714848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc0be5a0 0x7f84bc151cc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 -- 172.21.15.118:0/3409714848 >> 172.21.15.118:0/3409714848 conn(0x7f84bc06c020 msgr2=0x7f84bc0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.214 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 -- 172.21.15.118:0/3409714848 shutdown_connections 2024-08-18T13:52:00.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 -- 172.21.15.118:0/3409714848 wait complete. 2024-08-18T13:52:00.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.206+0000 7f84c0a1a700 1 Processor -- start 2024-08-18T13:52:00.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 -- start start 2024-08-18T13:52:00.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84bc0be5a0 0x7f84bc14c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc14b940 0x7f84bc165b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84bc14bd10 0x7f84bc166070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0c0df0 con 0x7f84bc0be5a0 2024-08-18T13:52:00.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc165630 con 0x7f84bc14b940 2024-08-18T13:52:00.218 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0c1d30 con 0x7f84bc14bd10 2024-08-18T13:52:00.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84bc0be5a0 0x7f84bc14c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84bc0be5a0 0x7f84bc14c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44970/0 (socket says 172.21.15.118:44970) 2024-08-18T13:52:00.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 -- 172.21.15.118:0/4167634252 learned_addr learned my addr 172.21.15.118:0/4167634252 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:00.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84baffd700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc14b940 0x7f84bc165b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.219 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bbfff700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84bc14bd10 0x7f84bc166070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 -- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84bc14bd10 msgr2=0x7f84bc166070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84bc14bd10 0x7f84bc166070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 -- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc14b940 msgr2=0x7f84bc165b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc14b940 0x7f84bc165b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 -- 172.21.15.118:0/4167634252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac009a20 con 0x7f84bc0be5a0 2024-08-18T13:52:00.220 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84bb7fe700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84bc0be5a0 0x7f84bc14c360 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c7c0 tx=0x7f84b0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84a37fe700 1 -- 172.21.15.118:0/4167634252 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0005b30 con 0x7f84bc0be5a0 2024-08-18T13:52:00.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84a37fe700 1 -- 172.21.15.118:0/4167634252 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b0005cb0 con 0x7f84bc0be5a0 2024-08-18T13:52:00.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc0c10b0 con 0x7f84bc0be5a0 2024-08-18T13:52:00.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84a37fe700 1 -- 172.21.15.118:0/4167634252 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007320 con 0x7f84bc0be5a0 2024-08-18T13:52:00.221 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc0c1670 con 0x7f84bc0be5a0 2024-08-18T13:52:00.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.210+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc071410 con 0x7f84bc0be5a0 2024-08-18T13:52:00.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.214+0000 7f84a37fe700 1 -- 172.21.15.118:0/4167634252 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f84b001b3b0 con 0x7f84bc0be5a0 2024-08-18T13:52:00.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.214+0000 7f84a37fe700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f84a8060860 0x7f84a8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.214+0000 7f84a37fe700 1 -- 172.21.15.118:0/4167634252 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f84b007c7a0 con 0x7f84bc0be5a0 2024-08-18T13:52:00.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.214+0000 7f84baffd700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f84a8060860 0x7f84a8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.214+0000 7f84baffd700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f84a8060860 0x7f84a8062d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0be9a0 tx=0x7f84ac019000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.218+0000 7f84a37fe700 1 -- 172.21.15.118:0/4167634252 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f84b004c5a0 con 0x7f84bc0be5a0 2024-08-18T13:52:00.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.382+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f84bc0c2a10 con 0x7f84a8060860 2024-08-18T13:52:00.386 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.382+0000 7f84a37fe700 1 -- 172.21.15.118:0/4167634252 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f84bc0c2a10 con 0x7f84a8060860 2024-08-18T13:52:00.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f84a8060860 msgr2=0x7f84a8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f84a8060860 0x7f84a8062d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0be9a0 tx=0x7f84ac019000 comp rx=0 tx=0).stop 2024-08-18T13:52:00.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84bc0be5a0 msgr2=0x7f84bc14c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.391 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84bc0be5a0 0x7f84bc14c360 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c7c0 tx=0x7f84b0007e00 comp rx=0 tx=0).stop 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 shutdown_connections 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f84a8060860 0x7f84a8062d10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84bc14bd10 0x7f84bc166070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84bc14b940 0x7f84bc165b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 --2- 172.21.15.118:0/4167634252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f84bc0be5a0 0x7f84bc14c360 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 >> 172.21.15.118:0/4167634252 conn(0x7f84bc06c020 msgr2=0x7f84bc151cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 shutdown_connections 2024-08-18T13:52:00.392 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.386+0000 7f84c0a1a700 1 -- 172.21.15.118:0/4167634252 wait complete. 2024-08-18T13:52:00.393 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 0 to deep-scrub 2024-08-18T13:52:00.405 INFO:tasks.ceph:Scrubbing osd.1 2024-08-18T13:52:00.405 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:00.515 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdafb03700 1 Processor -- start 2024-08-18T13:52:00.516 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdafb03700 1 -- start start 2024-08-18T13:52:00.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdafb03700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbda80bf7c0 0x7fbda814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdafb03700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 0x7fbda814c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdafb03700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda8155740 con 0x7fbda80c1fe0 2024-08-18T13:52:00.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdafb03700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda814c9c0 con 0x7fbda80bf7c0 2024-08-18T13:52:00.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdafb03700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda81565a0 con 0x7fbda805e6b0 2024-08-18T13:52:00.517 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdad89f700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbda80c1fe0 0x7fbda8067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38286/0 (socket says 172.21.15.118:38286) 2024-08-18T13:52:00.518 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.510+0000 7fbdad89f700 1 -- 172.21.15.118:0/1687143327 learned_addr learned my addr 172.21.15.118:0/1687143327 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:00.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533914309 0 0) 0x7fbda8155740 con 0x7fbda80c1fe0 2024-08-18T13:52:00.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd94003570 con 0x7fbda80c1fe0 2024-08-18T13:52:00.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007658293 0 0) 0x7fbd94003570 con 0x7fbda80c1fe0 2024-08-18T13:52:00.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 msgr2=0x7fbda814c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:00.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 0x7fbda814c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.522 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 >> v2:172.21.15.178:3300/0 conn(0x7fbda80bf7c0 msgr2=0x7fbda814bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:00.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 --2- 172.21.15.118:0/1687143327 >> v2:172.21.15.178:3300/0 conn(0x7fbda80bf7c0 0x7fbda814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda805ebc0 con 0x7fbda80c1fe0 2024-08-18T13:52:00.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fbd98001470 con 0x7fbda80c1fe0 2024-08-18T13:52:00.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 >> v1:172.21.15.118:6789/0 conn(0x7fbda80c1fe0 legacy=0x7fbda8067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd94003ae0 0x7fbd94003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 0x7fbd94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd94004f00 0x7fbd9400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda814c9c0 con 0x7fbd94003ae0 2024-08-18T13:52:00.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda81565a0 con 0x7fbd94004f00 2024-08-18T13:52:00.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94005690 con 0x7fbda805e6b0 2024-08-18T13:52:00.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd94003ae0 0x7fbd94003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad09e700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 0x7fbd94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad89f700 1 -- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 msgr2=0x7fbd94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 0x7fbd94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad89f700 1 -- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd94004f00 msgr2=0x7fbd9400ba60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd94004f00 0x7fbd9400ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad89f700 1 -- 172.21.15.118:0/1687143327 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94003570 con 0x7fbd94003ae0 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad09e700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbda805e6b0 0x7fbd94004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd94003ae0 0x7fbd94003ef0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c2d0 tx=0x7fbd98001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f51bcca816735e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800cba0 con 0x7fbd94003ae0 2024-08-18T13:52:00.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9800cd20 con 0x7fbd94003ae0 2024-08-18T13:52:00.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdac89d700 1 -- 172.21.15.118:0/1687143327 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98005b70 con 0x7fbd94003ae0 2024-08-18T13:52:00.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdafb03700 1 -- 172.21.15.118:0/1687143327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd94003ae0 msgr2=0x7fbd94003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdafb03700 1 --2- 172.21.15.118:0/1687143327 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbd94003ae0 0x7fbd94003ef0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800c2d0 tx=0x7fbd98001dc0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdae0a0700 1 -- 172.21.15.118:0/1687143327 reap_dead start 2024-08-18T13:52:00.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdafb03700 1 -- 172.21.15.118:0/1687143327 shutdown_connections 2024-08-18T13:52:00.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdafb03700 1 -- 172.21.15.118:0/1687143327 >> 172.21.15.118:0/1687143327 conn(0x7fbda80c8d60 msgr2=0x7fbda80c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdafb03700 1 -- 172.21.15.118:0/1687143327 shutdown_connections 2024-08-18T13:52:00.527 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.514+0000 7fbdafb03700 1 -- 172.21.15.118:0/1687143327 wait complete. 2024-08-18T13:52:00.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 Processor -- start 2024-08-18T13:52:00.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 -- start start 2024-08-18T13:52:00.530 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd94003ae0 0x7fbda8163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda8150530 0x7fbda8150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbda81528e0 0x7fbda8152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda806e290 con 0x7fbda81528e0 2024-08-18T13:52:00.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda81500a0 con 0x7fbda8150530 2024-08-18T13:52:00.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda8150ec0 con 0x7fbd94003ae0 2024-08-18T13:52:00.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbda81528e0 0x7fbda8152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.531 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbda81528e0 0x7fbda8152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:44994/0 (socket says 172.21.15.118:44994) 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 -- 172.21.15.118:0/1451322409 learned_addr learned my addr 172.21.15.118:0/1451322409 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdad09e700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda8150530 0x7fbda8150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd94003ae0 0x7fbda8163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 -- 172.21.15.118:0/1451322409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd94003ae0 msgr2=0x7fbda8163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd94003ae0 0x7fbda8163940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 -- 172.21.15.118:0/1451322409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda8150530 msgr2=0x7fbda8150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbda8150530 0x7fbda8150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 -- 172.21.15.118:0/1451322409 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda8153290 con 0x7fbda81528e0 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd94003ae0 0x7fbda8163940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdae0a0700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbda81528e0 0x7fbda8152d50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbda805eaf0 tx=0x7fbda0002f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0007030 con 0x7fbda81528e0 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0007190 con 0x7fbda81528e0 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.518+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda806e4e0 con 0x7fbda81528e0 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda00047e0 con 0x7fbda81528e0 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda806eaa0 con 0x7fbda81528e0 2024-08-18T13:52:00.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbda80648f0 con 0x7fbda81528e0 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbda0004c10 con 0x7fbda81528e0 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbd9dffb700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd8c060860 0x7fbd8c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fbda007c670 con 0x7fbda81528e0 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd8c060860 0x7fbd8c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbd9dffb700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7fbd8c06a5e0 0x7fbd8c06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 --> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd8c06d140 con 0x7fbd8c06a5e0 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbda007ecb0 con 0x7fbda81528e0 2024-08-18T13:52:00.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbdad09e700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7fbd8c06a5e0 0x7fbd8c06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbdad09e700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7fbd8c06a5e0 0x7fbd8c06ca70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.522+0000 7fbdad89f700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd8c060860 0x7fbd8c062d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd98015040 tx=0x7fbd98012000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.526+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== osd.1 v2:172.21.15.118:6816/24279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbd8c06d140 con 0x7fbd8c06a5e0 2024-08-18T13:52:00.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.538+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 --> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbda806cf00 con 0x7fbd8c06a5e0 2024-08-18T13:52:00.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbd9dffb700 1 -- 172.21.15.118:0/1451322409 <== osd.1 v2:172.21.15.118:6816/24279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbda806cf00 con 0x7fbd8c06a5e0 2024-08-18T13:52:00.546 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:00.546 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:00.546 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:00.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7fbd8c06a5e0 msgr2=0x7fbd8c06ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6816/24279,v1:172.21.15.118:6817/24279] conn(0x7fbd8c06a5e0 0x7fbd8c06ca70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd8c060860 msgr2=0x7fbd8c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbd8c060860 0x7fbd8c062d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd98015040 tx=0x7fbd98012000 comp rx=0 tx=0).stop 2024-08-18T13:52:00.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbda81528e0 msgr2=0x7fbda8152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 --2- 172.21.15.118:0/1451322409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbda81528e0 0x7fbda8152d50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbda805eaf0 tx=0x7fbda0002f20 comp rx=0 tx=0).stop 2024-08-18T13:52:00.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdae0a0700 1 -- 172.21.15.118:0/1451322409 reap_dead start 2024-08-18T13:52:00.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 shutdown_connections 2024-08-18T13:52:00.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 >> 172.21.15.118:0/1451322409 conn(0x7fbda80c8d60 msgr2=0x7fbda80c29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 shutdown_connections 2024-08-18T13:52:00.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.542+0000 7fbdafb03700 1 -- 172.21.15.118:0/1451322409 wait complete. 2024-08-18T13:52:00.558 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-18T13:52:00.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.662+0000 7fcc0cc9e700 1 Processor -- start 2024-08-18T13:52:00.669 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.662+0000 7fcc0cc9e700 1 -- start start 2024-08-18T13:52:00.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0cc9e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcc080c27d0 0x7fcc080673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0cc9e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc0814d850 0x7fcc0815dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.672 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0cc9e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0806f270 con 0x7fcc0806f700 2024-08-18T13:52:00.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0cc9e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc080c3520 con 0x7fcc080c27d0 2024-08-18T13:52:00.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0cc9e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0806e660 con 0x7fcc0814d850 2024-08-18T13:52:00.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0659c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcc080c27d0 0x7fcc080673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.673 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc06d9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc0814d850 0x7fcc0815dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc05d9b700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fcc0806f700 0x7fcc0814d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:55184/0 (socket says 172.21.15.118:55184) 2024-08-18T13:52:00.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc05d9b700 1 -- 172.21.15.118:0/3159963255 learned_addr learned my addr 172.21.15.118:0/3159963255 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:00.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0559a700 1 -- 172.21.15.118:0/3159963255 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340551279 0 0) 0x7fcc0806f270 con 0x7fcc0806f700 2024-08-18T13:52:00.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc06d9d700 1 -- 172.21.15.118:0/3159963255 >> v2:172.21.15.178:3300/0 conn(0x7fcc080c27d0 msgr2=0x7fcc080673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:00.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc06d9d700 1 --2- 172.21.15.118:0/3159963255 >> v2:172.21.15.178:3300/0 conn(0x7fcc080c27d0 0x7fcc080673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc06d9d700 1 -- 172.21.15.118:0/3159963255 >> v1:172.21.15.178:6789/0 conn(0x7fcc0806f700 legacy=0x7fcc0814d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc06d9d700 1 -- 172.21.15.118:0/3159963255 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0806e940 con 0x7fcc0814d850 2024-08-18T13:52:00.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0659c700 1 --2- 172.21.15.118:0/3159963255 >> v2:172.21.15.178:3300/0 conn(0x7fcc080c27d0 0x7fcc080673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:52:00.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc06d9d700 1 --2- 172.21.15.118:0/3159963255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc0814d850 0x7fcc0815dcc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098e0 tx=0x7fcc000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f4a21c15ab8323 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0559a700 1 -- 172.21.15.118:0/3159963255 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000b640 con 0x7fcc0814d850 2024-08-18T13:52:00.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0559a700 1 -- 172.21.15.118:0/3159963255 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000b7a0 con 0x7fcc0814d850 2024-08-18T13:52:00.676 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0559a700 1 -- 172.21.15.118:0/3159963255 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000bb40 con 0x7fcc0814d850 2024-08-18T13:52:00.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/3159963255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc0814d850 msgr2=0x7fcc0815dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.677 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.666+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/3159963255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc0814d850 0x7fcc0815dcc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098e0 tx=0x7fcc000091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/3159963255 shutdown_connections 2024-08-18T13:52:00.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/3159963255 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc0814d850 0x7fcc0815dcc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/3159963255 >> v2:172.21.15.178:3300/0 conn(0x7fcc080c27d0 0x7fcc080673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/3159963255 >> 172.21.15.118:0/3159963255 conn(0x7fcc080c8660 msgr2=0x7fcc080c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/3159963255 shutdown_connections 2024-08-18T13:52:00.678 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/3159963255 wait complete. 2024-08-18T13:52:00.682 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 Processor -- start 2024-08-18T13:52:00.682 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.670+0000 7fcc0cc9e700 1 -- start start 2024-08-18T13:52:00.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0806f700 0x7fcc08153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc080c27d0 0x7fcc081485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc0814d850 0x7fcc08148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08163c90 con 0x7fcc0806f700 2024-08-18T13:52:00.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0815a110 con 0x7fcc0814d850 2024-08-18T13:52:00.686 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081544c0 con 0x7fcc080c27d0 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0806f700 0x7fcc08153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0806f700 0x7fcc08153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45000/0 (socket says 172.21.15.118:45000) 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 -- 172.21.15.118:0/2425590710 learned_addr learned my addr 172.21.15.118:0/2425590710 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc06d9d700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc0814d850 0x7fcc08148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc05d9b700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc080c27d0 0x7fcc081485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 -- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc080c27d0 msgr2=0x7fcc081485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc080c27d0 0x7fcc081485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.687 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 -- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc0814d850 msgr2=0x7fcc08148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc0814d850 0x7fcc08148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 -- 172.21.15.118:0/2425590710 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0009420 con 0x7fcc0806f700 2024-08-18T13:52:00.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc05d9b700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc080c27d0 0x7fcc081485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:00.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0659c700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0806f700 0x7fcc08153670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000ec80 tx=0x7fcbf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcbfe7fc700 1 -- 172.21.15.118:0/2425590710 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000c8f0 con 0x7fcc0806f700 2024-08-18T13:52:00.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcbfe7fc700 1 -- 172.21.15.118:0/2425590710 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf000ca50 con 0x7fcc0806f700 2024-08-18T13:52:00.688 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc00009420 con 0x7fcc0806f700 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcbfe7fc700 1 -- 172.21.15.118:0/2425590710 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000cdd0 con 0x7fcc0806f700 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.674+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0815a600 con 0x7fcc0806f700 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.678+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc080642f0 con 0x7fcc0806f700 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.678+0000 7fcbfe7fc700 1 -- 172.21.15.118:0/2425590710 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcbf0013680 con 0x7fcc0806f700 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.678+0000 7fcbfe7fc700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbe8060860 0x7fcbe8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.678+0000 7fcc05d9b700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbe8060860 0x7fcbe8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.678+0000 7fcbfe7fc700 1 -- 172.21.15.118:0/2425590710 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fcbf007d590 con 0x7fcc0806f700 2024-08-18T13:52:00.689 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.678+0000 7fcc05d9b700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbe8060860 0x7fcbe8062d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc0806a0d0 tx=0x7fcbf8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.690 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.686+0000 7fcbfe7fc700 1 -- 172.21.15.118:0/2425590710 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcbf004d380 con 0x7fcc0806f700 2024-08-18T13:52:00.853 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.850+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc081594a0 con 0x7fcbe8060860 2024-08-18T13:52:00.854 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.850+0000 7fcbfe7fc700 1 -- 172.21.15.118:0/2425590710 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcc081594a0 con 0x7fcbe8060860 2024-08-18T13:52:00.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbe8060860 msgr2=0x7fcbe8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbe8060860 0x7fcbe8062d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc0806a0d0 tx=0x7fcbf8009040 comp rx=0 tx=0).stop 2024-08-18T13:52:00.858 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0806f700 msgr2=0x7fcc08153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0806f700 0x7fcc08153670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000ec80 tx=0x7fcbf00091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 shutdown_connections 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fcbe8060860 0x7fcbe8062d10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc0814d850 0x7fcc08148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc080c27d0 0x7fcc081485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 --2- 172.21.15.118:0/2425590710 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcc0806f700 0x7fcc08153670 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 >> 172.21.15.118:0/2425590710 conn(0x7fcc080c8660 msgr2=0x7fcc0814abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.859 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 shutdown_connections 2024-08-18T13:52:00.860 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.854+0000 7fcc0cc9e700 1 -- 172.21.15.118:0/2425590710 wait complete. 2024-08-18T13:52:00.860 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 1 to deep-scrub 2024-08-18T13:52:00.872 INFO:tasks.ceph:Scrubbing osd.2 2024-08-18T13:52:00.873 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:00.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f70d56700 1 Processor -- start 2024-08-18T13:52:00.986 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f70d56700 1 -- start start 2024-08-18T13:52:00.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f70d56700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c0c3b10 0x7f1f6c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.988 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f70d56700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f6c0c3ee0 0x7f1f6c067860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f70d56700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c0673d0 con 0x7f1f6c06bf70 2024-08-18T13:52:00.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f70d56700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c06f7a0 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f70d56700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0bea90 con 0x7f1f6c0c3b10 2024-08-18T13:52:00.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6bfff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f1f6c06bf70 0x7f1f6c14cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38296/0 (socket says 172.21.15.118:38296) 2024-08-18T13:52:00.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6bfff700 1 -- 172.21.15.118:0/1662426170 learned_addr learned my addr 172.21.15.118:0/1662426170 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:00.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6affd700 1 --2- 172.21.15.118:0/1662426170 >> v2:172.21.15.178:3300/0 conn(0x7f1f6c0c3ee0 0x7f1f6c067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.989 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6b7fe700 1 --2- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c0c3b10 0x7f1f6c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6a7fc700 1 -- 172.21.15.118:0/1662426170 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790353949 0 0) 0x7f1f6c0673d0 con 0x7f1f6c06bf70 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6a7fc700 1 -- 172.21.15.118:0/1662426170 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f58003570 con 0x7f1f6c06bf70 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6affd700 1 -- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c0c3b10 msgr2=0x7f1f6c06a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6affd700 1 --2- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c0c3b10 0x7f1f6c06a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6affd700 1 -- 172.21.15.118:0/1662426170 >> v1:172.21.15.118:6789/0 conn(0x7f1f6c06bf70 legacy=0x7f1f6c14cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6affd700 1 -- 172.21.15.118:0/1662426170 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c153190 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6affd700 1 --2- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c067860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009e60 tx=0x7f1f54009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7491c377c92aa881 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.990 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6a7fc700 1 -- 172.21.15.118:0/1662426170 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400e070 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6a7fc700 1 -- 172.21.15.118:0/1662426170 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5400bab0 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.991 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.982+0000 7f1f6a7fc700 1 -- 172.21.15.118:0/1662426170 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5400be30 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3ee0 msgr2=0x7f1f6c067860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.993 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 --2- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c067860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009e60 tx=0x7f1f54009780 comp rx=0 tx=0).stop 2024-08-18T13:52:00.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- 172.21.15.118:0/1662426170 shutdown_connections 2024-08-18T13:52:00.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 --2- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c067860 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 --2- 172.21.15.118:0/1662426170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c0c3b10 0x7f1f6c06a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- 172.21.15.118:0/1662426170 >> 172.21.15.118:0/1662426170 conn(0x7f1f6c06b1f0 msgr2=0x7f1f6c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:00.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- 172.21.15.118:0/1662426170 shutdown_connections 2024-08-18T13:52:00.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- 172.21.15.118:0/1662426170 wait complete. 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 Processor -- start 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- start start 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c06bf70 0x7f1f6c1461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3b10 0x7f1f6c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c156130 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0be8f0 con 0x7f1f6c0c3b10 2024-08-18T13:52:00.995 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f70d56700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c162f50 con 0x7f1f6c06bf70 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45006/0 (socket says 172.21.15.118:45006) 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 -- 172.21.15.118:0/2431281307 learned_addr learned my addr 172.21.15.118:0/2431281307 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6affd700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3b10 0x7f1f6c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6b7fe700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c06bf70 0x7f1f6c1461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 -- 172.21.15.118:0/2431281307 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c06bf70 msgr2=0x7f1f6c1461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f6c06bf70 0x7f1f6c1461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.996 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 -- 172.21.15.118:0/2431281307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3b10 msgr2=0x7f1f6c154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:00.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c0c3b10 0x7f1f6c154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:00.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.986+0000 7f1f6bfff700 1 -- 172.21.15.118:0/2431281307 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f54009a20 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f6bfff700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c154850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1f640010e0 tx=0x7f1f64007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:00.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64005c00 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f64005d60 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640072e0 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.997 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c163160 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c163720 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f6c071410 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1f64007690 con 0x7f1f6c0c3ee0 2024-08-18T13:52:00.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1f4c060890 0x7f1f4c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:00.999 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f1f6407c5e0 con 0x7f1f6c0c3ee0 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f6b7fe700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1f4c060890 0x7f1f4c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f1f4c06a590 0x7f1f4c06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 --> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f4c06d0f0 con 0x7f1f4c06a590 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f6407ec20 con 0x7f1f6c0c3ee0 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f6affd700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f1f4c06a590 0x7f1f4c06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.990+0000 7f1f6b7fe700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1f4c060890 0x7f1f4c062d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009db0 tx=0x7f1f5c0091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.994+0000 7f1f6affd700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f1f4c06a590 0x7f1f4c06ca20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.000 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:00.994+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== osd.2 v2:172.21.15.118:6800/24282 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1f4c06d0f0 con 0x7f1f4c06a590 2024-08-18T13:52:01.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 --> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1f6c0c22a0 con 0x7f1f4c06a590 2024-08-18T13:52:01.014 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f637fe700 1 -- 172.21.15.118:0/2431281307 <== osd.2 v2:172.21.15.118:6800/24282 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1f6c0c22a0 con 0x7f1f4c06a590 2024-08-18T13:52:01.015 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:01.015 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:01.015 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:01.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f1f4c06a590 msgr2=0x7f1f4c06ca20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6800/24282,v1:172.21.15.118:6801/24282] conn(0x7f1f4c06a590 0x7f1f4c06ca20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1f4c060890 msgr2=0x7f1f4c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.016 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1f4c060890 0x7f1f4c062d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009db0 tx=0x7f1f5c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f6c0c3ee0 msgr2=0x7f1f6c154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 --2- 172.21.15.118:0/2431281307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1f6c0c3ee0 0x7f1f6c154850 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1f640010e0 tx=0x7f1f64007dd0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f6bfff700 1 -- 172.21.15.118:0/2431281307 reap_dead start 2024-08-18T13:52:01.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 shutdown_connections 2024-08-18T13:52:01.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 >> 172.21.15.118:0/2431281307 conn(0x7f1f6c06b1f0 msgr2=0x7f1f6c159040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 shutdown_connections 2024-08-18T13:52:01.017 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.010+0000 7f1f70d56700 1 -- 172.21.15.118:0/2431281307 wait complete. 2024-08-18T13:52:01.028 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-18T13:52:01.144 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.138+0000 7f3cc56c7700 1 Processor -- start 2024-08-18T13:52:01.145 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.138+0000 7f3cc56c7700 1 -- start start 2024-08-18T13:52:01.149 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3cc00684e0 0x7f3cc0155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.150 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3cc00be4b0 0x7f3cc0156000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.150 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc00c1110 con 0x7f3cc01556f0 2024-08-18T13:52:01.150 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc00c0b90 con 0x7f3cc00be4b0 2024-08-18T13:52:01.150 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc00c1cb0 con 0x7f3cc00684e0 2024-08-18T13:52:01.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3cc00be4b0 0x7f3cc0156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cbffff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f3cc01556f0 0x7f3cc00c15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38300/0 (socket says 172.21.15.118:38300) 2024-08-18T13:52:01.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f3cc00be4b0 0x7f3cc0156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45010/0 (socket says 172.21.15.118:45010) 2024-08-18T13:52:01.151 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cb77fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3cc00684e0 0x7f3cc0155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.152 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1338985213 learned_addr learned my addr 172.21.15.118:0/1338985213 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:01.152 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1338985213 >> v2:172.21.15.178:3300/0 conn(0x7f3cc00684e0 msgr2=0x7f3cc0155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:01.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 --2- 172.21.15.118:0/1338985213 >> v2:172.21.15.178:3300/0 conn(0x7f3cc00684e0 0x7f3cc0155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1338985213 >> v1:172.21.15.118:6789/0 conn(0x7f3cc01556f0 legacy=0x7f3cc00c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1338985213 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc00c8060 con 0x7f3cc00be4b0 2024-08-18T13:52:01.153 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc4ec6700 1 --2- 172.21.15.118:0/1338985213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc00be4b0 0x7f3cc0156000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0065150 tx=0x7f3cb800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c8c1bf452f160f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cbf7fe700 1 -- 172.21.15.118:0/1338985213 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8010070 con 0x7f3cc00be4b0 2024-08-18T13:52:01.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cbf7fe700 1 -- 172.21.15.118:0/1338985213 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb800dde0 con 0x7f3cc00be4b0 2024-08-18T13:52:01.154 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cbf7fe700 1 -- 172.21.15.118:0/1338985213 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800b320 con 0x7f3cc00be4b0 2024-08-18T13:52:01.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1338985213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc00be4b0 msgr2=0x7f3cc0156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1338985213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc00be4b0 0x7f3cc0156000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0065150 tx=0x7f3cb800ba50 comp rx=0 tx=0).stop 2024-08-18T13:52:01.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1338985213 shutdown_connections 2024-08-18T13:52:01.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1338985213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc00be4b0 0x7f3cc0156000 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1338985213 >> v2:172.21.15.178:3300/0 conn(0x7f3cc00684e0 0x7f3cc0155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1338985213 >> 172.21.15.118:0/1338985213 conn(0x7f3cc006c020 msgr2=0x7f3cc00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1338985213 shutdown_connections 2024-08-18T13:52:01.156 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.142+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1338985213 wait complete. 2024-08-18T13:52:01.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 Processor -- start 2024-08-18T13:52:01.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 -- start start 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cc00684e0 0x7f3cc0141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cc00be4b0 0x7f3cc0163c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc01556f0 0x7f3cc013d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01582e0 con 0x7f3cc01556f0 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc0153120 con 0x7f3cc00684e0 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01421a0 con 0x7f3cc00be4b0 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc01556f0 0x7f3cc013d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.163 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc01556f0 0x7f3cc013d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45026/0 (socket says 172.21.15.118:45026) 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1718460439 learned_addr learned my addr 172.21.15.118:0/1718460439 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cb77fe700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cc00be4b0 0x7f3cc0163c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cbffff700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cc00684e0 0x7f3cc0141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cc00be4b0 msgr2=0x7f3cc0163c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cc00be4b0 0x7f3cc0163c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cc00684e0 msgr2=0x7f3cc0141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cc00684e0 0x7f3cc0141650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 -- 172.21.15.118:0/1718460439 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb800bcf0 con 0x7f3cc01556f0 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cbffff700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cc00684e0 0x7f3cc0141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc4ec6700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc01556f0 0x7f3cc013d160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3cc005eb30 tx=0x7f3cb801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cbcff9700 1 -- 172.21.15.118:0/1718460439 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800b4d0 con 0x7f3cc01556f0 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cbcff9700 1 -- 172.21.15.118:0/1718460439 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb801e620 con 0x7f3cc01556f0 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc0158510 con 0x7f3cc01556f0 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cbcff9700 1 -- 172.21.15.118:0/1718460439 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb801e9e0 con 0x7f3cc01556f0 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.146+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc00c5ed0 con 0x7f3cc01556f0 2024-08-18T13:52:01.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.150+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc0071410 con 0x7f3cc01556f0 2024-08-18T13:52:01.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.150+0000 7f3cbcff9700 1 -- 172.21.15.118:0/1718460439 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3cb8003040 con 0x7f3cc01556f0 2024-08-18T13:52:01.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.150+0000 7f3cbcff9700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3ca0060860 0x7f3ca0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.150+0000 7f3cbffff700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3ca0060860 0x7f3ca0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.150+0000 7f3cbcff9700 1 -- 172.21.15.118:0/1718460439 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f3cb80814b0 con 0x7f3cc01556f0 2024-08-18T13:52:01.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.150+0000 7f3cbffff700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3ca0060860 0x7f3ca0062d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0001780 tx=0x7f3cb0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.154+0000 7f3cbcff9700 1 -- 172.21.15.118:0/1718460439 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3cb8051320 con 0x7f3cc01556f0 2024-08-18T13:52:01.333 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.330+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cc006da80 con 0x7f3ca0060860 2024-08-18T13:52:01.334 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.330+0000 7f3cbcff9700 1 -- 172.21.15.118:0/1718460439 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3cc006da80 con 0x7f3ca0060860 2024-08-18T13:52:01.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3ca0060860 msgr2=0x7f3ca0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.338 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3ca0060860 0x7f3ca0062d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0001780 tx=0x7f3cb0009040 comp rx=0 tx=0).stop 2024-08-18T13:52:01.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc01556f0 msgr2=0x7f3cc013d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc01556f0 0x7f3cc013d160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3cc005eb30 tx=0x7f3cb801a000 comp rx=0 tx=0).stop 2024-08-18T13:52:01.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 shutdown_connections 2024-08-18T13:52:01.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f3ca0060860 0x7f3ca0062d10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cc01556f0 0x7f3cc013d160 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cc00be4b0 0x7f3cc0163c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.339 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 --2- 172.21.15.118:0/1718460439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cc00684e0 0x7f3cc0141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 >> 172.21.15.118:0/1718460439 conn(0x7f3cc006c020 msgr2=0x7f3cc006ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 shutdown_connections 2024-08-18T13:52:01.340 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.334+0000 7f3cc56c7700 1 -- 172.21.15.118:0/1718460439 wait complete. 2024-08-18T13:52:01.341 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 2 to deep-scrub 2024-08-18T13:52:01.353 INFO:tasks.ceph:Scrubbing osd.3 2024-08-18T13:52:01.353 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:01.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 Processor -- start 2024-08-18T13:52:01.465 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 -- start start 2024-08-18T13:52:01.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f878c0c0b00 0x7f878c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c068820 0x7f878c0c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f878c0c8d40 0x7f878c158860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878c0c2490 con 0x7f878c0c8d40 2024-08-18T13:52:01.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878c0c7980 con 0x7f878c0c0b00 2024-08-18T13:52:01.468 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c156280 con 0x7f878c068820 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f878c0c8d40 0x7f878c158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f878c0c8d40 0x7f878c158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45032/0 (socket says 172.21.15.118:45032) 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 -- 172.21.15.118:0/3163154375 learned_addr learned my addr 172.21.15.118:0/3163154375 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87915e4700 1 --2- 172.21.15.118:0/3163154375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c068820 0x7f878c0c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8791de5700 1 --2- 172.21.15.118:0/3163154375 >> v2:172.21.15.178:3300/0 conn(0x7f878c0c0b00 0x7f878c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 -- 172.21.15.118:0/3163154375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c068820 msgr2=0x7f878c0c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 --2- 172.21.15.118:0/3163154375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c068820 0x7f878c0c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.469 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 -- 172.21.15.118:0/3163154375 >> v2:172.21.15.178:3300/0 conn(0x7f878c0c0b00 msgr2=0x7f878c0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:01.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 --2- 172.21.15.118:0/3163154375 >> v2:172.21.15.178:3300/0 conn(0x7f878c0c0b00 0x7f878c0c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 -- 172.21.15.118:0/3163154375 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0c8350 con 0x7f878c0c8d40 2024-08-18T13:52:01.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f87925e6700 1 --2- 172.21.15.118:0/3163154375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c0c8d40 0x7f878c158860 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f878c06fb80 tx=0x7f8784009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0352aa1c4324cd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8790de3700 1 -- 172.21.15.118:0/3163154375 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878400e070 con 0x7f878c0c8d40 2024-08-18T13:52:01.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8790de3700 1 -- 172.21.15.118:0/3163154375 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878400b970 con 0x7f878c0c8d40 2024-08-18T13:52:01.470 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8790de3700 1 -- 172.21.15.118:0/3163154375 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840092e0 con 0x7f878c0c8d40 2024-08-18T13:52:01.471 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 -- 172.21.15.118:0/3163154375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c0c8d40 msgr2=0x7f878c158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 --2- 172.21.15.118:0/3163154375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c0c8d40 0x7f878c158860 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f878c06fb80 tx=0x7f8784009840 comp rx=0 tx=0).stop 2024-08-18T13:52:01.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 -- 172.21.15.118:0/3163154375 shutdown_connections 2024-08-18T13:52:01.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 --2- 172.21.15.118:0/3163154375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c0c8d40 0x7f878c158860 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 --2- 172.21.15.118:0/3163154375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c068820 0x7f878c0c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 --2- 172.21.15.118:0/3163154375 >> v2:172.21.15.178:3300/0 conn(0x7f878c0c0b00 0x7f878c0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.462+0000 7f8792de7700 1 -- 172.21.15.118:0/3163154375 >> 172.21.15.118:0/3163154375 conn(0x7f878c0cc630 msgr2=0x7f878c0c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.472 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 -- 172.21.15.118:0/3163154375 shutdown_connections 2024-08-18T13:52:01.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 -- 172.21.15.118:0/3163154375 wait complete. 2024-08-18T13:52:01.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 Processor -- start 2024-08-18T13:52:01.473 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 -- start start 2024-08-18T13:52:01.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c068820 0x7f878c165370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878c0c0b00 0x7f878c168730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.475 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c0c8d40 0x7f878c165c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0b0980 con 0x7f878c068820 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0cb830 con 0x7f878c0c0b00 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8792de7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0b0bd0 con 0x7f878c0c8d40 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8791de5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c068820 0x7f878c165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8791de5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c068820 0x7f878c165370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45038/0 (socket says 172.21.15.118:45038) 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8791de5700 1 -- 172.21.15.118:0/3547820515 learned_addr learned my addr 172.21.15.118:0/3547820515 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f87925e6700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c0c8d40 0x7f878c165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.476 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f87915e4700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878c0c0b00 0x7f878c168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8791de5700 1 -- 172.21.15.118:0/3547820515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c0c8d40 msgr2=0x7f878c165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8791de5700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f878c0c8d40 0x7f878c165c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8791de5700 1 -- 172.21.15.118:0/3547820515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878c0c0b00 msgr2=0x7f878c168730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.466+0000 7f8791de5700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f878c0c0b00 0x7f878c168730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8791de5700 1 -- 172.21.15.118:0/3547820515 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784009ae0 con 0x7f878c068820 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8791de5700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c068820 0x7f878c165370 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f878800ec80 tx=0x7f87880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800c9d0 con 0x7f878c068820 2024-08-18T13:52:01.477 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878800cb30 con 0x7f878c068820 2024-08-18T13:52:01.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c164890 con 0x7f878c068820 2024-08-18T13:52:01.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788013470 con 0x7f878c068820 2024-08-18T13:52:01.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c164e50 con 0x7f878c068820 2024-08-18T13:52:01.478 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f878c071410 con 0x7f878c068820 2024-08-18T13:52:01.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8788013820 con 0x7f878c068820 2024-08-18T13:52:01.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f8781ffb700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8770060860 0x7f8770062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.479 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.470+0000 7f87915e4700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8770060860 0x7f8770062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f878807d5a0 con 0x7f878c068820 2024-08-18T13:52:01.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f8781ffb700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7f877006a5e0 0x7f877006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 --> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f877006d140 con 0x7f877006a5e0 2024-08-18T13:52:01.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f878807fb00 con 0x7f878c068820 2024-08-18T13:52:01.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f87925e6700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7f877006a5e0 0x7f877006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f87915e4700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8770060860 0x7f8770062d10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f878c06a230 tx=0x7f877c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.480 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f87925e6700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7f877006a5e0 0x7f877006ca70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.481 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.474+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== osd.3 v2:172.21.15.118:6802/24280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f877006d140 con 0x7f877006a5e0 2024-08-18T13:52:01.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.490+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 --> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f878c0894a0 con 0x7f877006a5e0 2024-08-18T13:52:01.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8781ffb700 1 -- 172.21.15.118:0/3547820515 <== osd.3 v2:172.21.15.118:6802/24280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f878c0894a0 con 0x7f877006a5e0 2024-08-18T13:52:01.498 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:01.498 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:01.498 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:01.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7f877006a5e0 msgr2=0x7f877006ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6802/24280,v1:172.21.15.118:6804/24280] conn(0x7f877006a5e0 0x7f877006ca70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8770060860 msgr2=0x7f8770062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f8770060860 0x7f8770062d10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f878c06a230 tx=0x7f877c008040 comp rx=0 tx=0).stop 2024-08-18T13:52:01.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c068820 msgr2=0x7f878c165370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 --2- 172.21.15.118:0/3547820515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f878c068820 0x7f878c165370 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f878800ec80 tx=0x7f87880091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f87925e6700 1 -- 172.21.15.118:0/3547820515 reap_dead start 2024-08-18T13:52:01.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 shutdown_connections 2024-08-18T13:52:01.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 >> 172.21.15.118:0/3547820515 conn(0x7f878c0cc630 msgr2=0x7f878c05e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 shutdown_connections 2024-08-18T13:52:01.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.494+0000 7f8792de7700 1 -- 172.21.15.118:0/3547820515 wait complete. 2024-08-18T13:52:01.510 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-18T13:52:01.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.614+0000 7faa3e9ce700 1 Processor -- start 2024-08-18T13:52:01.620 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.614+0000 7faa3e9ce700 1 -- start start 2024-08-18T13:52:01.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faa381556f0 0x7faa380c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa380684e0 0x7faa38155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa380c1110 con 0x7faa380be4b0 2024-08-18T13:52:01.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa380c0b90 con 0x7faa381556f0 2024-08-18T13:52:01.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa380c8060 con 0x7faa380684e0 2024-08-18T13:52:01.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e1cd700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7faa380be4b0 0x7faa3814b7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:55194/0 (socket says 172.21.15.118:55194) 2024-08-18T13:52:01.622 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3d1cb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa380684e0 0x7faa38155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.623 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e1cd700 1 -- 172.21.15.118:0/3024907493 learned_addr learned my addr 172.21.15.118:0/3024907493 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:01.624 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3d9cc700 1 --2- 172.21.15.118:0/3024907493 >> v2:172.21.15.178:3300/0 conn(0x7faa381556f0 0x7faa380c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3c9ca700 1 -- 172.21.15.118:0/3024907493 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245487595 0 0) 0x7faa380c1110 con 0x7faa380be4b0 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3c9ca700 1 -- 172.21.15.118:0/3024907493 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa24003570 con 0x7faa380be4b0 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3d1cb700 1 -- 172.21.15.118:0/3024907493 >> v2:172.21.15.178:3300/0 conn(0x7faa381556f0 msgr2=0x7faa380c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3d1cb700 1 --2- 172.21.15.118:0/3024907493 >> v2:172.21.15.178:3300/0 conn(0x7faa381556f0 0x7faa380c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3d1cb700 1 -- 172.21.15.118:0/3024907493 >> v1:172.21.15.178:6789/0 conn(0x7faa380be4b0 legacy=0x7faa3814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3d1cb700 1 -- 172.21.15.118:0/3024907493 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380c8300 con 0x7faa380684e0 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3d1cb700 1 --2- 172.21.15.118:0/3024907493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa380684e0 0x7faa38155ac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009910 tx=0x7faa2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b60c0a66cb4bf4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.625 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3c9ca700 1 -- 172.21.15.118:0/3024907493 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00b640 con 0x7faa380684e0 2024-08-18T13:52:01.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3c9ca700 1 -- 172.21.15.118:0/3024907493 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c00b7a0 con 0x7faa380684e0 2024-08-18T13:52:01.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3c9ca700 1 -- 172.21.15.118:0/3024907493 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00bb60 con 0x7faa380684e0 2024-08-18T13:52:01.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3024907493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa380684e0 msgr2=0x7faa38155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3024907493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa380684e0 0x7faa38155ac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009910 tx=0x7faa2c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3024907493 shutdown_connections 2024-08-18T13:52:01.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3024907493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa380684e0 0x7faa38155ac0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.626 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3024907493 >> v2:172.21.15.178:3300/0 conn(0x7faa381556f0 0x7faa380c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3024907493 >> 172.21.15.118:0/3024907493 conn(0x7faa3806c020 msgr2=0x7faa380c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3024907493 shutdown_connections 2024-08-18T13:52:01.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.618+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3024907493 wait complete. 2024-08-18T13:52:01.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 Processor -- start 2024-08-18T13:52:01.627 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 -- start start 2024-08-18T13:52:01.628 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa380684e0 0x7faa3813e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa380be4b0 0x7faa3813eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa381556f0 0x7faa3806e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa38165f00 con 0x7faa380be4b0 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa38165a30 con 0x7faa380684e0 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e9ce700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa380c6540 con 0x7faa381556f0 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa380be4b0 0x7faa3813eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa380be4b0 0x7faa3813eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45050/0 (socket says 172.21.15.118:45050) 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 -- 172.21.15.118:0/3417976318 learned_addr learned my addr 172.21.15.118:0/3417976318 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:01.629 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3e1cd700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa381556f0 0x7faa3806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.630 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d9cc700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa380684e0 0x7faa3813e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 -- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa381556f0 msgr2=0x7faa3806e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa381556f0 0x7faa3806e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 -- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa380684e0 msgr2=0x7faa3813e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa380684e0 0x7faa3813e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.622+0000 7faa3d1cb700 1 -- 172.21.15.118:0/3417976318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c009420 con 0x7faa380be4b0 2024-08-18T13:52:01.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa3d1cb700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa380be4b0 0x7faa3813eeb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7faa380c0cd0 tx=0x7faa2c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.631 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa29ffb700 1 -- 172.21.15.118:0/3417976318 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c01f070 con 0x7faa380be4b0 2024-08-18T13:52:01.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa29ffb700 1 -- 172.21.15.118:0/3417976318 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c000bc0 con 0x7faa380be4b0 2024-08-18T13:52:01.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa38165c20 con 0x7faa380be4b0 2024-08-18T13:52:01.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa29ffb700 1 -- 172.21.15.118:0/3417976318 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00bb40 con 0x7faa380be4b0 2024-08-18T13:52:01.632 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa380c5ef0 con 0x7faa380be4b0 2024-08-18T13:52:01.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa380c66f0 con 0x7faa380be4b0 2024-08-18T13:52:01.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa29ffb700 1 -- 172.21.15.118:0/3417976318 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7faa2c023070 con 0x7faa380be4b0 2024-08-18T13:52:01.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa29ffb700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faa1c060860 0x7faa1c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa29ffb700 1 -- 172.21.15.118:0/3417976318 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7faa2c07e350 con 0x7faa380be4b0 2024-08-18T13:52:01.633 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.626+0000 7faa3d9cc700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faa1c060860 0x7faa1c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.634 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.630+0000 7faa3d9cc700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faa1c060860 0x7faa1c062d10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa34006fd0 tx=0x7faa34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.640 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.634+0000 7faa29ffb700 1 -- 172.21.15.118:0/3417976318 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faa380c66f0 con 0x7faa380be4b0 2024-08-18T13:52:01.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.798+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7faa380c66b0 con 0x7faa1c060860 2024-08-18T13:52:01.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.798+0000 7faa29ffb700 1 -- 172.21.15.118:0/3417976318 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faa380c66b0 con 0x7faa1c060860 2024-08-18T13:52:01.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faa1c060860 msgr2=0x7faa1c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faa1c060860 0x7faa1c062d10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa34006fd0 tx=0x7faa34008040 comp rx=0 tx=0).stop 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa380be4b0 msgr2=0x7faa3813eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa380be4b0 0x7faa3813eeb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7faa380c0cd0 tx=0x7faa2c017000 comp rx=0 tx=0).stop 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 shutdown_connections 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7faa1c060860 0x7faa1c062d10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa381556f0 0x7faa3806e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faa380be4b0 0x7faa3813eeb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 --2- 172.21.15.118:0/3417976318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa380684e0 0x7faa3813e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 >> 172.21.15.118:0/3417976318 conn(0x7faa3806c020 msgr2=0x7faa381458d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 shutdown_connections 2024-08-18T13:52:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.802+0000 7faa3e9ce700 1 -- 172.21.15.118:0/3417976318 wait complete. 2024-08-18T13:52:01.810 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 3 to deep-scrub 2024-08-18T13:52:01.824 INFO:tasks.ceph:Scrubbing osd.4 2024-08-18T13:52:01.824 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:01.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 Processor -- start 2024-08-18T13:52:01.940 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 -- start start 2024-08-18T13:52:01.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f36100c0b00 0x7f36100c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.942 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3610068820 0x7f36100c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f36100c8d40 0x7f3610158860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36100c2490 con 0x7f36100c8d40 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36100c7980 con 0x7f36100c0b00 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.934+0000 7f3615b87700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3610156280 con 0x7f3610068820 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f36100c8d40 0x7f3610158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f36100c8d40 0x7f3610158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45060/0 (socket says 172.21.15.118:45060) 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 -- 172.21.15.118:0/1498154069 learned_addr learned my addr 172.21.15.118:0/1498154069 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:01.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3614b85700 1 --2- 172.21.15.118:0/1498154069 >> v2:172.21.15.178:3300/0 conn(0x7f36100c0b00 0x7f36100c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 -- 172.21.15.118:0/1498154069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3610068820 msgr2=0x7f36100c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:01.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3607fff700 1 --2- 172.21.15.118:0/1498154069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3610068820 0x7f36100c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 --2- 172.21.15.118:0/1498154069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3610068820 0x7f36100c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 -- 172.21.15.118:0/1498154069 >> v2:172.21.15.178:3300/0 conn(0x7f36100c0b00 msgr2=0x7f36100c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:01.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 --2- 172.21.15.118:0/1498154069 >> v2:172.21.15.178:3300/0 conn(0x7f36100c0b00 0x7f36100c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 -- 172.21.15.118:0/1498154069 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100c8350 con 0x7f36100c8d40 2024-08-18T13:52:01.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3614b85700 1 --2- 172.21.15.118:0/1498154069 >> v2:172.21.15.178:3300/0 conn(0x7f36100c0b00 0x7f36100c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:01.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3607fff700 1 --2- 172.21.15.118:0/1498154069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3610068820 0x7f36100c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:52:01.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615386700 1 --2- 172.21.15.118:0/1498154069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f361006fb80 tx=0x7f3608009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58de35af65e0050 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f36077fe700 1 -- 172.21.15.118:0/1498154069 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800e070 con 0x7f36100c8d40 2024-08-18T13:52:01.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f36077fe700 1 -- 172.21.15.118:0/1498154069 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800ba50 con 0x7f36100c8d40 2024-08-18T13:52:01.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f36077fe700 1 -- 172.21.15.118:0/1498154069 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36080092e0 con 0x7f36100c8d40 2024-08-18T13:52:01.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 -- 172.21.15.118:0/1498154069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 msgr2=0x7f3610158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 --2- 172.21.15.118:0/1498154069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f361006fb80 tx=0x7f3608009840 comp rx=0 tx=0).stop 2024-08-18T13:52:01.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 -- 172.21.15.118:0/1498154069 shutdown_connections 2024-08-18T13:52:01.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 --2- 172.21.15.118:0/1498154069 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158860 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 --2- 172.21.15.118:0/1498154069 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3610068820 0x7f36100c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 --2- 172.21.15.118:0/1498154069 >> v2:172.21.15.178:3300/0 conn(0x7f36100c0b00 0x7f36100c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 -- 172.21.15.118:0/1498154069 >> 172.21.15.118:0/1498154069 conn(0x7f36100cc630 msgr2=0x7f36100c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 -- 172.21.15.118:0/1498154069 shutdown_connections 2024-08-18T13:52:01.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.938+0000 7f3615b87700 1 -- 172.21.15.118:0/1498154069 wait complete. 2024-08-18T13:52:01.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 Processor -- start 2024-08-18T13:52:01.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 -- start start 2024-08-18T13:52:01.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3610068820 0x7f36101489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36100c0b00 0x7f3610158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3610167db0 con 0x7f36100c8d40 2024-08-18T13:52:01.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36100b1b20 con 0x7f3610068820 2024-08-18T13:52:01.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36100b12f0 con 0x7f36100c0b00 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45070/0 (socket says 172.21.15.118:45070) 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 -- 172.21.15.118:0/3731744778 learned_addr learned my addr 172.21.15.118:0/3731744778 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3614b85700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3610068820 0x7f36101489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 -- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36100c0b00 msgr2=0x7f3610158050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36100c0b00 0x7f3610158050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 -- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3610068820 msgr2=0x7f36101489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.951 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3610068820 0x7f36101489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 -- 172.21.15.118:0/3731744778 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608009ae0 con 0x7f36100c8d40 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3614b85700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3610068820 0x7f36101489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615386700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f361006fbe0 tx=0x7f3608018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36080032f0 con 0x7f36100c8d40 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608008e40 con 0x7f36100c8d40 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36101590b0 con 0x7f36100c8d40 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360801c640 con 0x7f36100c8d40 2024-08-18T13:52:01.952 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.942+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36100b1560 con 0x7f36100c8d40 2024-08-18T13:52:01.953 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3610071410 con 0x7f36100c8d40 2024-08-18T13:52:01.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3608003040 con 0x7f36100c8d40 2024-08-18T13:52:01.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3604ff9700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f35fc060860 0x7f35fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f360807ef10 con 0x7f36100c8d40 2024-08-18T13:52:01.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3614b85700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f35fc060860 0x7f35fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3604ff9700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f35fc06a5e0 0x7f35fc06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:01.954 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 --> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35fc06d140 con 0x7f35fc06a5e0 2024-08-18T13:52:01.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f360807f200 con 0x7f36100c8d40 2024-08-18T13:52:01.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.946+0000 7f3614b85700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f35fc060860 0x7f35fc062d10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f360c006f00 tx=0x7f360c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.950+0000 7f3607fff700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f35fc06a5e0 0x7f35fc06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:01.955 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.950+0000 7f3607fff700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f35fc06a5e0 0x7f35fc06ca70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:01.956 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.950+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== osd.4 v2:172.21.15.178:6808/24441 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f35fc06d140 con 0x7f35fc06a5e0 2024-08-18T13:52:01.974 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 --> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f36100894a0 con 0x7f35fc06a5e0 2024-08-18T13:52:01.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3604ff9700 1 -- 172.21.15.118:0/3731744778 <== osd.4 v2:172.21.15.178:6808/24441 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f36100894a0 con 0x7f35fc06a5e0 2024-08-18T13:52:01.975 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:01.976 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:01.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f35fc06a5e0 msgr2=0x7f35fc06ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.178:6808/24441,v1:172.21.15.178:6809/24441] conn(0x7f35fc06a5e0 0x7f35fc06ca70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:01.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f35fc060860 msgr2=0x7f35fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f35fc060860 0x7f35fc062d10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f360c006f00 tx=0x7f360c008040 comp rx=0 tx=0).stop 2024-08-18T13:52:01.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 msgr2=0x7f3610158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:01.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 --2- 172.21.15.118:0/3731744778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f36100c8d40 0x7f3610158590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f361006fbe0 tx=0x7f3608018000 comp rx=0 tx=0).stop 2024-08-18T13:52:01.977 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615386700 1 -- 172.21.15.118:0/3731744778 reap_dead start 2024-08-18T13:52:01.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 shutdown_connections 2024-08-18T13:52:01.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 >> 172.21.15.118:0/3731744778 conn(0x7f36100cc630 msgr2=0x7f3610069680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:01.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 shutdown_connections 2024-08-18T13:52:01.978 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:01.970+0000 7f3615b87700 1 -- 172.21.15.118:0/3731744778 wait complete. 2024-08-18T13:52:01.988 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-18T13:52:02.100 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.094+0000 7f448d748700 1 Processor -- start 2024-08-18T13:52:02.101 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.094+0000 7f448d748700 1 -- start start 2024-08-18T13:52:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f448814c2a0 0x7f4488069580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.102 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4488152b30 0x7f448806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44880690f0 con 0x7f44880c1340 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44880c07b0 con 0x7f4488152b30 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4488152fc0 con 0x7f448814c2a0 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4488152b30 0x7f448806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f4486ffd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f448814c2a0 0x7f4488069580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4488152b30 0x7f448806c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45086/0 (socket says 172.21.15.118:45086) 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 -- 172.21.15.118:0/3256334114 learned_addr learned my addr 172.21.15.118:0/3256334114 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:02.103 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 -- 172.21.15.118:0/3256334114 >> v2:172.21.15.178:3300/0 conn(0x7f448814c2a0 msgr2=0x7f4488069580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:02.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 --2- 172.21.15.118:0/3256334114 >> v2:172.21.15.178:3300/0 conn(0x7f448814c2a0 0x7f4488069580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 -- 172.21.15.118:0/3256334114 >> v1:172.21.15.178:6789/0 conn(0x7f44880c1340 legacy=0x7f4488155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 -- 172.21.15.118:0/3256334114 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44880687c0 con 0x7f4488152b30 2024-08-18T13:52:02.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f44867fc700 1 --2- 172.21.15.118:0/3256334114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4488152b30 0x7f448806c4f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f448805ec80 tx=0x7f4470009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df01febffbf358bd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f4485ffb700 1 -- 172.21.15.118:0/3256334114 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009e70 con 0x7f4488152b30 2024-08-18T13:52:02.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f4485ffb700 1 -- 172.21.15.118:0/3256334114 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447000b860 con 0x7f4488152b30 2024-08-18T13:52:02.104 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f4485ffb700 1 -- 172.21.15.118:0/3256334114 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000bc00 con 0x7f4488152b30 2024-08-18T13:52:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- 172.21.15.118:0/3256334114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4488152b30 msgr2=0x7f448806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.107 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 --2- 172.21.15.118:0/3256334114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4488152b30 0x7f448806c4f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f448805ec80 tx=0x7f4470009250 comp rx=0 tx=0).stop 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- 172.21.15.118:0/3256334114 shutdown_connections 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 --2- 172.21.15.118:0/3256334114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4488152b30 0x7f448806c4f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 --2- 172.21.15.118:0/3256334114 >> v2:172.21.15.178:3300/0 conn(0x7f448814c2a0 0x7f4488069580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- 172.21.15.118:0/3256334114 >> 172.21.15.118:0/3256334114 conn(0x7f448806f530 msgr2=0x7f44880c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- 172.21.15.118:0/3256334114 shutdown_connections 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- 172.21.15.118:0/3256334114 wait complete. 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 Processor -- start 2024-08-18T13:52:02.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.098+0000 7f448d748700 1 -- start start 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44880c1340 0x7f448814daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f448814c2a0 0x7f4488151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4488152b30 0x7f448813b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44881532d0 con 0x7f44880c1340 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448814f1f0 con 0x7f4488152b30 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4488153570 con 0x7f448814c2a0 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44880c1340 0x7f448814daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44880c1340 0x7f448814daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45094/0 (socket says 172.21.15.118:45094) 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 -- 172.21.15.118:0/3876580678 learned_addr learned my addr 172.21.15.118:0/3876580678 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f44877fe700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4488152b30 0x7f448813b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f44867fc700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f448814c2a0 0x7f4488151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 -- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f448814c2a0 msgr2=0x7f4488151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f448814c2a0 0x7f4488151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 -- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4488152b30 msgr2=0x7f448813b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4488152b30 0x7f448813b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 -- 172.21.15.118:0/3876580678 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44700094f0 con 0x7f44880c1340 2024-08-18T13:52:02.110 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f4486ffd700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44880c1340 0x7f448814daa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4480009f20 tx=0x7f448000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f447effd700 1 -- 172.21.15.118:0/3876580678 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000cb70 con 0x7f44880c1340 2024-08-18T13:52:02.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f447effd700 1 -- 172.21.15.118:0/3876580678 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448000f040 con 0x7f44880c1340 2024-08-18T13:52:02.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448814ce70 con 0x7f44880c1340 2024-08-18T13:52:02.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f447effd700 1 -- 172.21.15.118:0/3876580678 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480014480 con 0x7f44880c1340 2024-08-18T13:52:02.111 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448814f720 con 0x7f44880c1340 2024-08-18T13:52:02.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.102+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448814f910 con 0x7f44880c1340 2024-08-18T13:52:02.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.106+0000 7f447effd700 1 -- 172.21.15.118:0/3876580678 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f44800147f0 con 0x7f44880c1340 2024-08-18T13:52:02.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.106+0000 7f447effd700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4468060890 0x7f4468062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.106+0000 7f447effd700 1 -- 172.21.15.118:0/3876580678 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f448007daa0 con 0x7f44880c1340 2024-08-18T13:52:02.112 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.106+0000 7f44867fc700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4468060890 0x7f4468062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.106+0000 7f44867fc700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4468060890 0x7f4468062d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f448805ec80 tx=0x7f4470017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.116 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.110+0000 7f447effd700 1 -- 172.21.15.118:0/3876580678 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f448004d930 con 0x7f44880c1340 2024-08-18T13:52:02.285 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f44881536b0 con 0x7f4468060890 2024-08-18T13:52:02.286 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f447effd700 1 -- 172.21.15.118:0/3876580678 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f447000b9d0 con 0x7f4468060890 2024-08-18T13:52:02.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4468060890 msgr2=0x7f4468062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4468060890 0x7f4468062d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f448805ec80 tx=0x7f4470017000 comp rx=0 tx=0).stop 2024-08-18T13:52:02.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44880c1340 msgr2=0x7f448814daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44880c1340 0x7f448814daa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4480009f20 tx=0x7f448000b040 comp rx=0 tx=0).stop 2024-08-18T13:52:02.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 shutdown_connections 2024-08-18T13:52:02.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4468060890 0x7f4468062d40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.289 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4488152b30 0x7f448813b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f448814c2a0 0x7f4488151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 --2- 172.21.15.118:0/3876580678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44880c1340 0x7f448814daa0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 >> 172.21.15.118:0/3876580678 conn(0x7f448806f530 msgr2=0x7f4488149930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 shutdown_connections 2024-08-18T13:52:02.290 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.282+0000 7f448d748700 1 -- 172.21.15.118:0/3876580678 wait complete. 2024-08-18T13:52:02.291 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 4 to deep-scrub 2024-08-18T13:52:02.301 INFO:tasks.ceph:Scrubbing osd.5 2024-08-18T13:52:02.301 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:02.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 Processor -- start 2024-08-18T13:52:02.412 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 -- start start 2024-08-18T13:52:02.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1da40c87c0 0x7f1da4068e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 0x7f1da40693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1da40c8b90 0x7f1da40c0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.413 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da4156190 con 0x7f1da40c8b90 2024-08-18T13:52:02.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da40689e0 con 0x7f1da40c87c0 2024-08-18T13:52:02.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da963b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4155630 con 0x7f1da41551a0 2024-08-18T13:52:02.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.406+0000 7f1da8e3a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1da40c8b90 0x7f1da40c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f1da40c8b90 0x7f1da40c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45110/0 (socket says 172.21.15.118:45110) 2024-08-18T13:52:02.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 -- 172.21.15.118:0/3043339553 learned_addr learned my addr 172.21.15.118:0/3043339553 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:02.414 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da3fff700 1 --2- 172.21.15.118:0/3043339553 >> v2:172.21.15.178:3300/0 conn(0x7f1da40c87c0 0x7f1da4068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 -- 172.21.15.118:0/3043339553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 msgr2=0x7f1da40693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:02.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da37fe700 1 --2- 172.21.15.118:0/3043339553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 0x7f1da40693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 --2- 172.21.15.118:0/3043339553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 0x7f1da40693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 -- 172.21.15.118:0/3043339553 >> v2:172.21.15.178:3300/0 conn(0x7f1da40c87c0 msgr2=0x7f1da4068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:02.416 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 --2- 172.21.15.118:0/3043339553 >> v2:172.21.15.178:3300/0 conn(0x7f1da40c87c0 0x7f1da4068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 -- 172.21.15.118:0/3043339553 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da41558f0 con 0x7f1da40c8b90 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da8e3a700 1 --2- 172.21.15.118:0/3043339553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c8b90 0x7f1da40c0930 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1da40699c0 tx=0x7f1d9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e077971394e45e85 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da2ffd700 1 -- 172.21.15.118:0/3043339553 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00e070 con 0x7f1da40c8b90 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da2ffd700 1 -- 172.21.15.118:0/3043339553 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00bab0 con 0x7f1da40c8b90 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da2ffd700 1 -- 172.21.15.118:0/3043339553 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0092e0 con 0x7f1da40c8b90 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 -- 172.21.15.118:0/3043339553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c8b90 msgr2=0x7f1da40c0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 --2- 172.21.15.118:0/3043339553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c8b90 0x7f1da40c0930 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1da40699c0 tx=0x7f1d9c009840 comp rx=0 tx=0).stop 2024-08-18T13:52:02.417 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 -- 172.21.15.118:0/3043339553 shutdown_connections 2024-08-18T13:52:02.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 --2- 172.21.15.118:0/3043339553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c8b90 0x7f1da40c0930 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 --2- 172.21.15.118:0/3043339553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 0x7f1da40693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 --2- 172.21.15.118:0/3043339553 >> v2:172.21.15.178:3300/0 conn(0x7f1da40c87c0 0x7f1da4068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 -- 172.21.15.118:0/3043339553 >> 172.21.15.118:0/3043339553 conn(0x7f1da40673d0 msgr2=0x7f1da40677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 -- 172.21.15.118:0/3043339553 shutdown_connections 2024-08-18T13:52:02.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.410+0000 7f1da963b700 1 -- 172.21.15.118:0/3043339553 wait complete. 2024-08-18T13:52:02.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 Processor -- start 2024-08-18T13:52:02.420 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 -- start start 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c87c0 0x7f1da414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da40c8b90 0x7f1da415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 0x7f1da414ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4165e60 con 0x7f1da40c87c0 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da414f1c0 con 0x7f1da40c8b90 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4165000 con 0x7f1da41551a0 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c87c0 0x7f1da414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c87c0 0x7f1da414e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45116/0 (socket says 172.21.15.118:45116) 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 -- 172.21.15.118:0/2448382436 learned_addr learned my addr 172.21.15.118:0/2448382436 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da37fe700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da40c8b90 0x7f1da415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da8e3a700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 0x7f1da414ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 -- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 msgr2=0x7f1da414ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da41551a0 0x7f1da414ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 -- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da40c8b90 msgr2=0x7f1da415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da40c8b90 0x7f1da415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.422 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 -- 172.21.15.118:0/2448382436 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c009ae0 con 0x7f1da40c87c0 2024-08-18T13:52:02.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da37fe700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da40c8b90 0x7f1da415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:02.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da3fff700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c87c0 0x7f1da414e470 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800ec80 tx=0x7f1d980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800c850 con 0x7f1da40c87c0 2024-08-18T13:52:02.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9800c9b0 con 0x7f1da40c87c0 2024-08-18T13:52:02.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da41652e0 con 0x7f1da40c87c0 2024-08-18T13:52:02.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800cd50 con 0x7f1da40c87c0 2024-08-18T13:52:02.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da41658a0 con 0x7f1da40c87c0 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.414+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1da4071410 con 0x7f1da40c87c0 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1d98013680 con 0x7f1da40c87c0 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1d8bfff700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1d90060860 0x7f1d90062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f1d9807d820 con 0x7f1da40c87c0 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1d8bfff700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f1d9006a5e0 0x7f1d9006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 --> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d9006d140 con 0x7f1d9006a5e0 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d9807db10 con 0x7f1da40c87c0 2024-08-18T13:52:02.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1da37fe700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1d90060860 0x7f1d90062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1da8e3a700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f1d9006a5e0 0x7f1d9006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1da37fe700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1d90060860 0x7f1d90062d10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006fd0 tx=0x7f1d94008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.418+0000 7f1da8e3a700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f1d9006a5e0 0x7f1d9006ca70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.426 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.422+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== osd.5 v2:172.21.15.178:6817/24442 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1d9006d140 con 0x7f1d9006a5e0 2024-08-18T13:52:02.444 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.438+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 --> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1da406fcc0 con 0x7f1d9006a5e0 2024-08-18T13:52:02.444 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.438+0000 7f1d8bfff700 1 -- 172.21.15.118:0/2448382436 <== osd.5 v2:172.21.15.178:6817/24442 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1da406fcc0 con 0x7f1d9006a5e0 2024-08-18T13:52:02.445 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:02.445 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:02.445 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:02.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f1d9006a5e0 msgr2=0x7f1d9006ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.178:6817/24442,v1:172.21.15.178:6820/24442] conn(0x7f1d9006a5e0 0x7f1d9006ca70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.446 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1d90060860 msgr2=0x7f1d90062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1d90060860 0x7f1d90062d10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d94006fd0 tx=0x7f1d94008040 comp rx=0 tx=0).stop 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c87c0 msgr2=0x7f1da414e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 --2- 172.21.15.118:0/2448382436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1da40c87c0 0x7f1da414e470 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800ec80 tx=0x7f1d980091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da8e3a700 1 -- 172.21.15.118:0/2448382436 reap_dead start 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 shutdown_connections 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 >> 172.21.15.118:0/2448382436 conn(0x7f1da40673d0 msgr2=0x7f1da41537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 shutdown_connections 2024-08-18T13:52:02.447 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.442+0000 7f1da963b700 1 -- 172.21.15.118:0/2448382436 wait complete. 2024-08-18T13:52:02.457 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-18T13:52:02.566 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbefb200700 1 Processor -- start 2024-08-18T13:52:02.567 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbefb200700 1 -- start start 2024-08-18T13:52:02.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbefb200700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbef414bd30 0x7fbef41590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbefb200700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef405ed70 con 0x7fbef40c5820 2024-08-18T13:52:02.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbefb200700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef405e830 con 0x7fbef4155840 2024-08-18T13:52:02.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbefb200700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef405ead0 con 0x7fbef414bd30 2024-08-18T13:52:02.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbef99fd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fbef40c5820 0x7fbef4155130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38306/0 (socket says 172.21.15.118:38306) 2024-08-18T13:52:02.569 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbef99fd700 1 -- 172.21.15.118:0/1773115394 learned_addr learned my addr 172.21.15.118:0/1773115394 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:02.570 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.562+0000 7fbefa1fe700 1 --2- 172.21.15.118:0/1773115394 >> v2:172.21.15.178:3300/0 conn(0x7fbef414bd30 0x7fbef41590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742185697 0 0) 0x7fbef405ed70 con 0x7fbef40c5820 2024-08-18T13:52:02.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee0003570 con 0x7fbef40c5820 2024-08-18T13:52:02.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061407986 0 0) 0x7fbef405e830 con 0x7fbef4155840 2024-08-18T13:52:02.571 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef405ed70 con 0x7fbef4155840 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefa1fe700 1 -- 172.21.15.118:0/1773115394 >> v1:172.21.15.178:6789/0 conn(0x7fbef4155840 legacy=0x7fbef40c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefa1fe700 1 -- 172.21.15.118:0/1773115394 >> v1:172.21.15.118:6789/0 conn(0x7fbef40c5820 legacy=0x7fbef4155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3391922498 0 0) 0x7fbee0003570 con 0x7fbef40c5820 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefa1fe700 1 -- 172.21.15.118:0/1773115394 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4068650 con 0x7fbef414bd30 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefa1fe700 1 --2- 172.21.15.118:0/1773115394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef414bd30 0x7fbef41590f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbef00098e0 tx=0x7fbef00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77b1f5e6aa725cd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4001470 con 0x7fbef414bd30 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef0011420 con 0x7fbef414bd30 2024-08-18T13:52:02.572 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbef91fc700 1 -- 172.21.15.118:0/1773115394 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef00117a0 con 0x7fbef414bd30 2024-08-18T13:52:02.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefb200700 1 -- 172.21.15.118:0/1773115394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef414bd30 msgr2=0x7fbef41590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.573 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefb200700 1 --2- 172.21.15.118:0/1773115394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef414bd30 0x7fbef41590f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbef00098e0 tx=0x7fbef00091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefb200700 1 -- 172.21.15.118:0/1773115394 shutdown_connections 2024-08-18T13:52:02.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefb200700 1 --2- 172.21.15.118:0/1773115394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef414bd30 0x7fbef41590f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefb200700 1 -- 172.21.15.118:0/1773115394 >> 172.21.15.118:0/1773115394 conn(0x7fbef406a8b0 msgr2=0x7fbef406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefb200700 1 -- 172.21.15.118:0/1773115394 shutdown_connections 2024-08-18T13:52:02.574 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.566+0000 7fbefb200700 1 -- 172.21.15.118:0/1773115394 wait complete. 2024-08-18T13:52:02.575 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 Processor -- start 2024-08-18T13:52:02.575 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 -- start start 2024-08-18T13:52:02.577 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef40c5820 0x7fbef4156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef414bd30 0x7fbef4157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbef4155840 0x7fbef4146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4156a90 con 0x7fbef4155840 2024-08-18T13:52:02.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef414c8c0 con 0x7fbef40c5820 2024-08-18T13:52:02.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.570+0000 7fbefb200700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c1700 con 0x7fbef414bd30 2024-08-18T13:52:02.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbef4155840 0x7fbef4146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.578 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbef4155840 0x7fbef4146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45118/0 (socket says 172.21.15.118:45118) 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 -- 172.21.15.118:0/607965571 learned_addr learned my addr 172.21.15.118:0/607965571 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa1fe700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef40c5820 0x7fbef4156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbef99fd700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef414bd30 0x7fbef4157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 -- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef414bd30 msgr2=0x7fbef4157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef414bd30 0x7fbef4157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 -- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef40c5820 msgr2=0x7fbef4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef40c5820 0x7fbef4156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 -- 172.21.15.118:0/607965571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0009420 con 0x7fbef4155840 2024-08-18T13:52:02.579 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa9ff700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbef4155840 0x7fbef4146890 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c860 tx=0x7fbeec002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbeea7fc700 1 -- 172.21.15.118:0/607965571 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec002030 con 0x7fbef4155840 2024-08-18T13:52:02.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbeea7fc700 1 -- 172.21.15.118:0/607965571 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec0021b0 con 0x7fbef4155840 2024-08-18T13:52:02.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef40bfff0 con 0x7fbef4155840 2024-08-18T13:52:02.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbeea7fc700 1 -- 172.21.15.118:0/607965571 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec012630 con 0x7fbef4155840 2024-08-18T13:52:02.581 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef4156720 con 0x7fbef4155840 2024-08-18T13:52:02.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbeea7fc700 1 -- 172.21.15.118:0/607965571 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbeec012a20 con 0x7fbef4155840 2024-08-18T13:52:02.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef406fcc0 con 0x7fbef4155840 2024-08-18T13:52:02.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbeea7fc700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbed8060890 0x7fbed8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbeea7fc700 1 -- 172.21.15.118:0/607965571 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fbeec07d020 con 0x7fbef4155840 2024-08-18T13:52:02.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.574+0000 7fbefa1fe700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbed8060890 0x7fbed8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.582 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.578+0000 7fbefa1fe700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbed8060890 0x7fbed8062d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbef00093f0 tx=0x7fbef0016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.587 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.582+0000 7fbeea7fc700 1 -- 172.21.15.118:0/607965571 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbef406fcc0 con 0x7fbef4155840 2024-08-18T13:52:02.748 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.742+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef4156bd0 con 0x7fbed8060890 2024-08-18T13:52:02.749 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.742+0000 7fbeea7fc700 1 -- 172.21.15.118:0/607965571 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbef4156bd0 con 0x7fbed8060890 2024-08-18T13:52:02.751 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbed8060890 msgr2=0x7fbed8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.751 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbed8060890 0x7fbed8062d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbef00093f0 tx=0x7fbef0016000 comp rx=0 tx=0).stop 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbef4155840 msgr2=0x7fbef4146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbef4155840 0x7fbef4146890 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c860 tx=0x7fbeec002700 comp rx=0 tx=0).stop 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 shutdown_connections 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fbed8060890 0x7fbed8062d40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbef4155840 0x7fbef4146890 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbef414bd30 0x7fbef4157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 --2- 172.21.15.118:0/607965571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef40c5820 0x7fbef4156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.752 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 >> 172.21.15.118:0/607965571 conn(0x7fbef406a8b0 msgr2=0x7fbef40c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 shutdown_connections 2024-08-18T13:52:02.753 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.746+0000 7fbefb200700 1 -- 172.21.15.118:0/607965571 wait complete. 2024-08-18T13:52:02.753 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 5 to deep-scrub 2024-08-18T13:52:02.763 INFO:tasks.ceph:Scrubbing osd.6 2024-08-18T13:52:02.763 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:02.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.870+0000 7f752d1eb700 1 Processor -- start 2024-08-18T13:52:02.877 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.870+0000 7f752d1eb700 1 -- start start 2024-08-18T13:52:02.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75280c0b00 0x7f75280c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f75280c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75280c7980 con 0x7f75280c8d40 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75280c2490 con 0x7f75280c0b00 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7528156280 con 0x7f7528068820 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f75280c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f75280c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:43690/0 (socket says 172.21.15.118:43690) 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 -- 172.21.15.118:0/3573542384 learned_addr learned my addr 172.21.15.118:0/3573542384 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f7527fff700 1 --2- 172.21.15.118:0/3573542384 >> v2:172.21.15.178:3300/0 conn(0x7f75280c0b00 0x7f75280c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 -- 172.21.15.118:0/3573542384 >> v2:172.21.15.178:3300/0 conn(0x7f75280c0b00 msgr2=0x7f75280c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:02.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 --2- 172.21.15.118:0/3573542384 >> v2:172.21.15.178:3300/0 conn(0x7f75280c0b00 0x7f75280c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 -- 172.21.15.118:0/3573542384 >> v1:172.21.15.178:6789/0 conn(0x7f75280c8d40 legacy=0x7f7528158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 -- 172.21.15.118:0/3573542384 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280c8350 con 0x7f7528068820 2024-08-18T13:52:02.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f75277fe700 1 --2- 172.21.15.118:0/3573542384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f75280c0ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f75180099b0 tx=0x7f7518009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf5f2e9e669c94 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f7526ffd700 1 -- 172.21.15.118:0/3573542384 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518009ea0 con 0x7f7528068820 2024-08-18T13:52:02.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f7526ffd700 1 -- 172.21.15.118:0/3573542384 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751800b860 con 0x7f7528068820 2024-08-18T13:52:02.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f7526ffd700 1 -- 172.21.15.118:0/3573542384 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751800bc00 con 0x7f7528068820 2024-08-18T13:52:02.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 -- 172.21.15.118:0/3573542384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 msgr2=0x7f75280c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 --2- 172.21.15.118:0/3573542384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f75280c0ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f75180099b0 tx=0x7f7518009250 comp rx=0 tx=0).stop 2024-08-18T13:52:02.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.874+0000 7f752d1eb700 1 -- 172.21.15.118:0/3573542384 shutdown_connections 2024-08-18T13:52:02.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 --2- 172.21.15.118:0/3573542384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f75280c0ed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 --2- 172.21.15.118:0/3573542384 >> v2:172.21.15.178:3300/0 conn(0x7f75280c0b00 0x7f75280c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 -- 172.21.15.118:0/3573542384 >> 172.21.15.118:0/3573542384 conn(0x7f75280cc630 msgr2=0x7f75280c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 -- 172.21.15.118:0/3573542384 shutdown_connections 2024-08-18T13:52:02.884 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 -- 172.21.15.118:0/3573542384 wait complete. 2024-08-18T13:52:02.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 Processor -- start 2024-08-18T13:52:02.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 -- start start 2024-08-18T13:52:02.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f7528151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75280c0b00 0x7f75281655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75280c8d40 0x7f7528149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7528169280 con 0x7f75280c0b00 2024-08-18T13:52:02.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75280c5cf0 con 0x7f75280c8d40 2024-08-18T13:52:02.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752d1eb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75280c6240 con 0x7f7528068820 2024-08-18T13:52:02.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75280c0b00 0x7f75281655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75280c0b00 0x7f75281655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45134/0 (socket says 172.21.15.118:45134) 2024-08-18T13:52:02.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 -- 172.21.15.118:0/2804120463 learned_addr learned my addr 172.21.15.118:0/2804120463 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:02.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f7527fff700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f7528151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f752c9ea700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75280c8d40 0x7f7528149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 -- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 msgr2=0x7f7528151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7528068820 0x7f7528151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 -- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75280c8d40 msgr2=0x7f7528149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75280c8d40 0x7f7528149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.878+0000 7f75277fe700 1 -- 172.21.15.118:0/2804120463 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180094f0 con 0x7f75280c0b00 2024-08-18T13:52:02.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f75277fe700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75280c0b00 0x7f75281655a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7518006fd0 tx=0x7f7518017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518000d10 con 0x7f75280c0b00 2024-08-18T13:52:02.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751800bbe0 con 0x7f75280c0b00 2024-08-18T13:52:02.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75280c5f60 con 0x7f75280c0b00 2024-08-18T13:52:02.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518011670 con 0x7f75280c0b00 2024-08-18T13:52:02.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7528168f10 con 0x7f75280c0b00 2024-08-18T13:52:02.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7528071410 con 0x7f75280c0b00 2024-08-18T13:52:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7518011a60 con 0x7f75280c0b00 2024-08-18T13:52:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7508060890 0x7f7508062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f751807f1f0 con 0x7f75280c0b00 2024-08-18T13:52:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f7527fff700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7508060890 0x7f7508062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f750806a4a0 0x7f750806c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 --> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f750806d000 con 0x7f750806a4a0 2024-08-18T13:52:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f751807f5c0 con 0x7f75280c0b00 2024-08-18T13:52:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f752c9ea700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f750806a4a0 0x7f750806c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.882+0000 7f7527fff700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7508060890 0x7f7508062d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7510009db0 tx=0x7f75100091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.886+0000 7f752c9ea700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f750806a4a0 0x7f750806c930 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:02.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.886+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== osd.6 v2:172.21.15.178:6816/24448 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f750806d000 con 0x7f750806a4a0 2024-08-18T13:52:02.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 --> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f75280c6930 con 0x7f750806a4a0 2024-08-18T13:52:02.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f751ffff700 1 -- 172.21.15.118:0/2804120463 <== osd.6 v2:172.21.15.178:6816/24448 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f75280c6930 con 0x7f750806a4a0 2024-08-18T13:52:02.906 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:02.907 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:02.907 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:02.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f750806a4a0 msgr2=0x7f750806c930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.178:6816/24448,v1:172.21.15.178:6818/24448] conn(0x7f750806a4a0 0x7f750806c930 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7508060890 msgr2=0x7f7508062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f7508060890 0x7f7508062d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7510009db0 tx=0x7f75100091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:02.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75280c0b00 msgr2=0x7f75281655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:02.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 --2- 172.21.15.118:0/2804120463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75280c0b00 0x7f75281655a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7518006fd0 tx=0x7f7518017000 comp rx=0 tx=0).stop 2024-08-18T13:52:02.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752c9ea700 1 -- 172.21.15.118:0/2804120463 reap_dead start 2024-08-18T13:52:02.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 shutdown_connections 2024-08-18T13:52:02.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 >> 172.21.15.118:0/2804120463 conn(0x7f75280cc630 msgr2=0x7f752805de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:02.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 shutdown_connections 2024-08-18T13:52:02.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:02.902+0000 7f752d1eb700 1 -- 172.21.15.118:0/2804120463 wait complete. 2024-08-18T13:52:02.924 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-08-18T13:52:03.037 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985d08700 1 Processor -- start 2024-08-18T13:52:03.038 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985d08700 1 -- start start 2024-08-18T13:52:03.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985d08700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 0x7fd9800673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985d08700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fd9800c3e40 0x7fd98014d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985d08700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98006cc90 con 0x7fd9800c3a50 2024-08-18T13:52:03.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985d08700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9800c83a0 con 0x7fd9800c3e40 2024-08-18T13:52:03.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985d08700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd980153090 con 0x7fd9800c3680 2024-08-18T13:52:03.042 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd977fff700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fd9800c3a50 0x7fd98014cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38310/0 (socket says 172.21.15.118:38310) 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd977fff700 1 -- 172.21.15.118:0/2031441098 learned_addr learned my addr 172.21.15.118:0/2031441098 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985507700 1 --2- 172.21.15.118:0/2031441098 >> v2:172.21.15.118:3300/0 conn(0x7fd9800c3e40 0x7fd98014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 0x7fd9800673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243294605 0 0) 0x7fd98006cc90 con 0x7fd9800c3a50 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd968003570 con 0x7fd9800c3a50 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 264566475 0 0) 0x7fd968003570 con 0x7fd9800c3a50 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 msgr2=0x7fd9800673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.043 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 0x7fd9800673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 >> v2:172.21.15.118:3300/0 conn(0x7fd9800c3e40 msgr2=0x7fd98014d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 --2- 172.21.15.118:0/2031441098 >> v2:172.21.15.118:3300/0 conn(0x7fd9800c3e40 0x7fd98014d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9800c8660 con 0x7fd9800c3a50 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fd970001470 con 0x7fd9800c3a50 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985507700 1 --2- 172.21.15.118:0/2031441098 >> v2:172.21.15.118:3300/0 conn(0x7fd9800c3e40 0x7fd98014d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 >> v1:172.21.15.118:6789/0 conn(0x7fd9800c3a50 legacy=0x7fd98014cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd968003a60 0x7fd968003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.044 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd968004810 0x7fd968004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 0x7fd9680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9800c83a0 con 0x7fd968003a60 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980153090 con 0x7fd968004810 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96800ba60 con 0x7fd9800c3680 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd968003a60 0x7fd968003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd977fff700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd968004810 0x7fd968004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd985507700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 0x7fd9680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 -- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 msgr2=0x7fd9680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.045 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9800c3680 0x7fd9680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 -- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd968004810 msgr2=0x7fd968004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd968004810 0x7fd968004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 -- 172.21.15.118:0/2031441098 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968003570 con 0x7fd968003a60 2024-08-18T13:52:03.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd984d06700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd968003a60 0x7fd968003e30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009c00 tx=0x7fd97c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c0085d25cbbece server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00e710 con 0x7fd968003a60 2024-08-18T13:52:03.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00e870 con 0x7fd968003a60 2024-08-18T13:52:03.046 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.034+0000 7fd9777fe700 1 -- 172.21.15.118:0/2031441098 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00ec30 con 0x7fd968003a60 2024-08-18T13:52:03.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- 172.21.15.118:0/2031441098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd968003a60 msgr2=0x7fd968003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 --2- 172.21.15.118:0/2031441098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd968003a60 0x7fd968003e30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009c00 tx=0x7fd97c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985507700 1 -- 172.21.15.118:0/2031441098 reap_dead start 2024-08-18T13:52:03.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- 172.21.15.118:0/2031441098 shutdown_connections 2024-08-18T13:52:03.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- 172.21.15.118:0/2031441098 >> 172.21.15.118:0/2031441098 conn(0x7fd98006c020 msgr2=0x7fd9800c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:03.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- 172.21.15.118:0/2031441098 shutdown_connections 2024-08-18T13:52:03.047 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- 172.21.15.118:0/2031441098 wait complete. 2024-08-18T13:52:03.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 Processor -- start 2024-08-18T13:52:03.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- start start 2024-08-18T13:52:03.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd968003a60 0x7fd980163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9801443a0 0x7fd980167150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.050 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd98014ebd0 0x7fd9800c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980167690 con 0x7fd9801443a0 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980140230 con 0x7fd98014ebd0 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd985d08700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd980163930 con 0x7fd968003a60 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd977fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9801443a0 0x7fd980167150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd977fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9801443a0 0x7fd980167150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45150/0 (socket says 172.21.15.118:45150) 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.038+0000 7fd977fff700 1 -- 172.21.15.118:0/900507222 learned_addr learned my addr 172.21.15.118:0/900507222 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd985507700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd98014ebd0 0x7fd9800c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.051 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd984d06700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd968003a60 0x7fd980163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd977fff700 1 -- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd968003a60 msgr2=0x7fd980163dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd977fff700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd968003a60 0x7fd980163dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd977fff700 1 -- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd98014ebd0 msgr2=0x7fd9800c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd977fff700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd98014ebd0 0x7fd9800c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd977fff700 1 -- 172.21.15.118:0/900507222 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9801455c0 con 0x7fd9801443a0 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd985507700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd98014ebd0 0x7fd9800c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd977fff700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9801443a0 0x7fd980167150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd970007430 tx=0x7fd970000d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd974ff9700 1 -- 172.21.15.118:0/900507222 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97000e860 con 0x7fd9801443a0 2024-08-18T13:52:03.052 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd974ff9700 1 -- 172.21.15.118:0/900507222 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97000e9e0 con 0x7fd9801443a0 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd980145880 con 0x7fd9801443a0 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd974ff9700 1 -- 172.21.15.118:0/900507222 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97000eda0 con 0x7fd9801443a0 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9801632e0 con 0x7fd9801443a0 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd980071410 con 0x7fd9801443a0 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd974ff9700 1 -- 172.21.15.118:0/900507222 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd9700133e0 con 0x7fd9801443a0 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd974ff9700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd96c060890 0x7fd96c062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd974ff9700 1 -- 172.21.15.118:0/900507222 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fd97007ce90 con 0x7fd9801443a0 2024-08-18T13:52:03.053 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.042+0000 7fd984d06700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd96c060890 0x7fd96c062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.046+0000 7fd984d06700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd96c060890 0x7fd96c062d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0091e0 tx=0x7fd97c01a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.046+0000 7fd974ff9700 1 -- 172.21.15.118:0/900507222 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd97004cd20 con 0x7fd9801443a0 2024-08-18T13:52:03.212 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.206+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fd980163a70 con 0x7fd96c060890 2024-08-18T13:52:03.213 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd974ff9700 1 -- 172.21.15.118:0/900507222 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd980163a70 con 0x7fd96c060890 2024-08-18T13:52:03.215 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd96c060890 msgr2=0x7fd96c062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd96c060890 0x7fd96c062d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0091e0 tx=0x7fd97c01a000 comp rx=0 tx=0).stop 2024-08-18T13:52:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9801443a0 msgr2=0x7fd980167150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9801443a0 0x7fd980167150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd970007430 tx=0x7fd970000d70 comp rx=0 tx=0).stop 2024-08-18T13:52:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 shutdown_connections 2024-08-18T13:52:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd96c060890 0x7fd96c062d40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd98014ebd0 0x7fd9800c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.216 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd9801443a0 0x7fd980167150 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 --2- 172.21.15.118:0/900507222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd968003a60 0x7fd980163dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 >> 172.21.15.118:0/900507222 conn(0x7fd98006c020 msgr2=0x7fd980148660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 shutdown_connections 2024-08-18T13:52:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.210+0000 7fd985d08700 1 -- 172.21.15.118:0/900507222 wait complete. 2024-08-18T13:52:03.217 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 6 to deep-scrub 2024-08-18T13:52:03.227 INFO:tasks.ceph:Scrubbing osd.7 2024-08-18T13:52:03.228 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-08-18T13:52:03.344 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.338+0000 7fd2450ae700 1 Processor -- start 2024-08-18T13:52:03.345 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.338+0000 7fd2450ae700 1 -- start start 2024-08-18T13:52:03.354 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.338+0000 7fd2450ae700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd240154f20 0x7fd24006a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.354 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2401552f0 0x7fd2400c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.354 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2400c1ce0 con 0x7fd240155750 2024-08-18T13:52:03.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd240068620 con 0x7fd240154f20 2024-08-18T13:52:03.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd240067ae0 con 0x7fd2401552f0 2024-08-18T13:52:03.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2448ad700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fd240155750 0x7fd2400673d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38312/0 (socket says 172.21.15.118:38312) 2024-08-18T13:52:03.355 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2448ad700 1 -- 172.21.15.118:0/1956859998 learned_addr learned my addr 172.21.15.118:0/1956859998 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:03.356 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23ffff700 1 --2- 172.21.15.118:0/1956859998 >> v2:172.21.15.178:3300/0 conn(0x7fd240154f20 0x7fd24006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.356 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23f7fe700 1 --2- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2401552f0 0x7fd2400c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.356 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23effd700 1 -- 172.21.15.118:0/1956859998 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514432663 0 0) 0x7fd2400c1ce0 con 0x7fd240155750 2024-08-18T13:52:03.357 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23effd700 1 -- 172.21.15.118:0/1956859998 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003570 con 0x7fd240155750 2024-08-18T13:52:03.357 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23ffff700 1 -- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2401552f0 msgr2=0x7fd2400c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.357 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23effd700 1 -- 172.21.15.118:0/1956859998 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755523082 0 0) 0x7fd22c003570 con 0x7fd240155750 2024-08-18T13:52:03.357 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23ffff700 1 --2- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2401552f0 0x7fd2400c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23ffff700 1 -- 172.21.15.118:0/1956859998 >> v1:172.21.15.118:6789/0 conn(0x7fd240155750 legacy=0x7fd2400673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23ffff700 1 -- 172.21.15.118:0/1956859998 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240068810 con 0x7fd240154f20 2024-08-18T13:52:03.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23ffff700 1 --2- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd240154f20 0x7fd24006a110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd238009e90 tx=0x7fd238009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c75cfcb636358b98 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23effd700 1 -- 172.21.15.118:0/1956859998 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800e070 con 0x7fd240154f20 2024-08-18T13:52:03.358 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23effd700 1 -- 172.21.15.118:0/1956859998 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23800b900 con 0x7fd240154f20 2024-08-18T13:52:03.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd23effd700 1 -- 172.21.15.118:0/1956859998 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230001470 con 0x7fd240154f20 2024-08-18T13:52:03.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd240154f20 msgr2=0x7fd24006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 --2- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd240154f20 0x7fd24006a110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd238009e90 tx=0x7fd238009780 comp rx=0 tx=0).stop 2024-08-18T13:52:03.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- 172.21.15.118:0/1956859998 shutdown_connections 2024-08-18T13:52:03.359 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 --2- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2401552f0 0x7fd2400c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 --2- 172.21.15.118:0/1956859998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd240154f20 0x7fd24006a110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- 172.21.15.118:0/1956859998 >> 172.21.15.118:0/1956859998 conn(0x7fd24006b1f0 msgr2=0x7fd24006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:03.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- 172.21.15.118:0/1956859998 shutdown_connections 2024-08-18T13:52:03.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.342+0000 7fd2450ae700 1 -- 172.21.15.118:0/1956859998 wait complete. 2024-08-18T13:52:03.360 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 Processor -- start 2024-08-18T13:52:03.361 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 -- start start 2024-08-18T13:52:03.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd240154f20 0x7fd24014e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2401552f0 0x7fd240162780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd240155750 0x7fd240142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24014f5e0 con 0x7fd240155750 2024-08-18T13:52:03.366 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2401666d0 con 0x7fd2401552f0 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd240166b60 con 0x7fd240154f20 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd240155750 0x7fd240142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd240155750 0x7fd240142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45162/0 (socket says 172.21.15.118:45162) 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 -- 172.21.15.118:0/4273760520 learned_addr learned my addr 172.21.15.118:0/4273760520 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd23f7fe700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2401552f0 0x7fd240162780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd23ffff700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd240154f20 0x7fd24014e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 -- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd240154f20 msgr2=0x7fd24014e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.367 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd240154f20 0x7fd24014e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 -- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2401552f0 msgr2=0x7fd240162780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2401552f0 0x7fd240162780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 -- 172.21.15.118:0/4273760520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238009a20 con 0x7fd240155750 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd23ffff700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd240154f20 0x7fd24014e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2448ad700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd240155750 0x7fd240142e50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd23000cb90 tx=0x7fd230001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230001d10 con 0x7fd240155750 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd230007360 con 0x7fd240155750 2024-08-18T13:52:03.368 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd240150a40 con 0x7fd240155750 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230004830 con 0x7fd240155750 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.346+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24014f810 con 0x7fd240155750 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd240071410 con 0x7fd240155750 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd230004ba0 con 0x7fd240155750 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd237fff700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd220060860 0x7fd220062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fd23007c680 con 0x7fd240155750 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd23ffff700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd220060860 0x7fd220062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.369 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd237fff700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7fd22006a560 0x7fd22006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 --> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd22006d0c0 con 0x7fd22006a560 2024-08-18T13:52:03.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd23007ebe0 con 0x7fd240155750 2024-08-18T13:52:03.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd23f7fe700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7fd22006a560 0x7fd22006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.350+0000 7fd23ffff700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd220060860 0x7fd220062d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd2380097c0 tx=0x7fd238017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.354+0000 7fd23f7fe700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7fd22006a560 0x7fd22006c9f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.370 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.354+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== osd.7 v2:172.21.15.178:6800/24440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fd22006d0c0 con 0x7fd22006a560 2024-08-18T13:52:03.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.370+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 --> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd2400673d0 con 0x7fd22006a560 2024-08-18T13:52:03.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.370+0000 7fd237fff700 1 -- 172.21.15.118:0/4273760520 <== osd.7 v2:172.21.15.178:6800/24440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd2400673d0 con 0x7fd22006a560 2024-08-18T13:52:03.377 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-08-18T13:52:03.377 INFO:teuthology.orchestra.run.smithi118.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-18T13:52:03.377 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-08-18T13:52:03.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7fd22006a560 msgr2=0x7fd22006c9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.178:6800/24440,v1:172.21.15.178:6801/24440] conn(0x7fd22006a560 0x7fd22006c9f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.378 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd220060860 msgr2=0x7fd220062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fd220060860 0x7fd220062d10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd2380097c0 tx=0x7fd238017000 comp rx=0 tx=0).stop 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd240155750 msgr2=0x7fd240142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 --2- 172.21.15.118:0/4273760520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd240155750 0x7fd240142e50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd23000cb90 tx=0x7fd230001aa0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2448ad700 1 -- 172.21.15.118:0/4273760520 reap_dead start 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 shutdown_connections 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 >> 172.21.15.118:0/4273760520 conn(0x7fd24006b1f0 msgr2=0x7fd2400be500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 shutdown_connections 2024-08-18T13:52:03.379 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.374+0000 7fd2450ae700 1 -- 172.21.15.118:0/4273760520 wait complete. 2024-08-18T13:52:03.390 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-08-18T13:52:03.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 Processor -- start 2024-08-18T13:52:03.494 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- start start 2024-08-18T13:52:03.496 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe1cc0af690 0x7fe1cc153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc0ca690 0x7fe1cc153e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1cc06d580 0x7fe1cc0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc1534a0 con 0x7fe1cc0af690 2024-08-18T13:52:03.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc156900 con 0x7fe1cc06d580 2024-08-18T13:52:03.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc156ba0 con 0x7fe1cc0ca690 2024-08-18T13:52:03.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe1cc0af690 0x7fe1cc153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.497 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7fe1cc0af690 0x7fe1cc153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45172/0 (socket says 172.21.15.118:45172) 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 -- 172.21.15.118:0/2831764379 learned_addr learned my addr 172.21.15.118:0/2831764379 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cb7fe700 1 --2- 172.21.15.118:0/2831764379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc0ca690 0x7fe1cc153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d0da6700 1 --2- 172.21.15.118:0/2831764379 >> v2:172.21.15.178:3300/0 conn(0x7fe1cc06d580 0x7fe1cc0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 -- 172.21.15.118:0/2831764379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc0ca690 msgr2=0x7fe1cc153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 --2- 172.21.15.118:0/2831764379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc0ca690 0x7fe1cc153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 -- 172.21.15.118:0/2831764379 >> v2:172.21.15.178:3300/0 conn(0x7fe1cc06d580 msgr2=0x7fe1cc0bea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 --2- 172.21.15.118:0/2831764379 >> v2:172.21.15.178:3300/0 conn(0x7fe1cc06d580 0x7fe1cc0bea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.498 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 -- 172.21.15.118:0/2831764379 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc156e80 con 0x7fe1cc0af690 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cb7fe700 1 --2- 172.21.15.118:0/2831764379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc0ca690 0x7fe1cc153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1cbfff700 1 --2- 172.21.15.118:0/2831764379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc153930 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc05ecd0 tx=0x7fe1c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d2041790d0a222 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1caffd700 1 -- 172.21.15.118:0/2831764379 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b860 con 0x7fe1cc0af690 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1caffd700 1 -- 172.21.15.118:0/2831764379 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000b9c0 con 0x7fe1cc0af690 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1caffd700 1 -- 172.21.15.118:0/2831764379 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000bd60 con 0x7fe1cc0af690 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- 172.21.15.118:0/2831764379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 msgr2=0x7fe1cc153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/2831764379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc153930 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc05ecd0 tx=0x7fe1c0009250 comp rx=0 tx=0).stop 2024-08-18T13:52:03.499 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- 172.21.15.118:0/2831764379 shutdown_connections 2024-08-18T13:52:03.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/2831764379 >> v2:172.21.15.178:3300/0 conn(0x7fe1cc06d580 0x7fe1cc0bea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/2831764379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc0ca690 0x7fe1cc153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/2831764379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc153930 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- 172.21.15.118:0/2831764379 >> 172.21.15.118:0/2831764379 conn(0x7fe1cc0c9a20 msgr2=0x7fe1cc0c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:03.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- 172.21.15.118:0/2831764379 shutdown_connections 2024-08-18T13:52:03.500 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.490+0000 7fe1d15a7700 1 -- 172.21.15.118:0/2831764379 wait complete. 2024-08-18T13:52:03.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 Processor -- start 2024-08-18T13:52:03.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 -- start start 2024-08-18T13:52:03.503 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc06d580 0x7fe1cc165c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc0ca690 0x7fe1cc0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0691d0 con 0x7fe1cc0af690 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0aa320 con 0x7fe1cc0ca690 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc068bc0 con 0x7fe1cc06d580 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc0ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45182/0 (socket says 172.21.15.118:45182) 2024-08-18T13:52:03.504 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 -- 172.21.15.118:0/1948732571 learned_addr learned my addr 172.21.15.118:0/1948732571 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d0da6700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc0ca690 0x7fe1cc0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cbfff700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc06d580 0x7fe1cc165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 -- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc06d580 msgr2=0x7fe1cc165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc06d580 0x7fe1cc165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 -- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc0ca690 msgr2=0x7fe1cc0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc0ca690 0x7fe1cc0adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 -- 172.21.15.118:0/1948732571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00094f0 con 0x7fe1cc0af690 2024-08-18T13:52:03.505 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1cb7fe700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc0ada60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e8d0 tx=0x7fe1bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1b3fff700 1 -- 172.21.15.118:0/1948732571 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc013410 con 0x7fe1cc0af690 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1b3fff700 1 -- 172.21.15.118:0/1948732571 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc013570 con 0x7fe1cc0af690 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0acc70 con 0x7fe1cc0af690 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1b3fff700 1 -- 172.21.15.118:0/1948732571 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc013910 con 0x7fe1cc0af690 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.494+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc0ad210 con 0x7fe1cc0af690 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.498+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc071410 con 0x7fe1cc0af690 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.498+0000 7fe1b3fff700 1 -- 172.21.15.118:0/1948732571 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe1bc013d40 con 0x7fe1cc0af690 2024-08-18T13:52:03.506 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.498+0000 7fe1b3fff700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe1b8060890 0x7fe1b8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.507 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.498+0000 7fe1b3fff700 1 -- 172.21.15.118:0/1948732571 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fe1bc07ce30 con 0x7fe1cc0af690 2024-08-18T13:52:03.507 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.498+0000 7fe1cbfff700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe1b8060890 0x7fe1b8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.507 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.498+0000 7fe1cbfff700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe1b8060890 0x7fe1b8062d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc05ecd0 tx=0x7fe1c0017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.508 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.502+0000 7fe1b3fff700 1 -- 172.21.15.118:0/1948732571 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe1bc04ccc0 con 0x7fe1cc0af690 2024-08-18T13:52:03.670 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.666+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1cc0c4500 con 0x7fe1b8060890 2024-08-18T13:52:03.671 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.666+0000 7fe1b3fff700 1 -- 172.21.15.118:0/1948732571 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe1cc0c4500 con 0x7fe1b8060890 2024-08-18T13:52:03.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe1b8060890 msgr2=0x7fe1b8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe1b8060890 0x7fe1b8062d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc05ecd0 tx=0x7fe1c0017000 comp rx=0 tx=0).stop 2024-08-18T13:52:03.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 msgr2=0x7fe1cc0ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc0ada60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e8d0 tx=0x7fe1bc008040 comp rx=0 tx=0).stop 2024-08-18T13:52:03.674 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 shutdown_connections 2024-08-18T13:52:03.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fe1b8060890 0x7fe1b8062d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1cc0ca690 0x7fe1cc0adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe1cc0af690 0x7fe1cc0ada60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 --2- 172.21.15.118:0/1948732571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1cc06d580 0x7fe1cc165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 >> 172.21.15.118:0/1948732571 conn(0x7fe1cc0c9a20 msgr2=0x7fe1cc0c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:03.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 shutdown_connections 2024-08-18T13:52:03.675 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.670+0000 7fe1d15a7700 1 -- 172.21.15.118:0/1948732571 wait complete. 2024-08-18T13:52:03.676 INFO:teuthology.orchestra.run.smithi118.stderr:instructed osd(s) 7 to deep-scrub 2024-08-18T13:52:03.686 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:52:03.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.790+0000 7f2d31814700 1 Processor -- start 2024-08-18T13:52:03.797 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.790+0000 7f2d31814700 1 -- start start 2024-08-18T13:52:03.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2d2c14bcd0 0x7f2d2c152410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c0bd580 0x7f2d2c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c151f80 con 0x7f2d2c0bea90 2024-08-18T13:52:03.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c151990 con 0x7f2d2c14bcd0 2024-08-18T13:52:03.799 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c14c0a0 con 0x7f2d2c0bd580 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2b7fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2d2c0bea90 0x7f2d2c06e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:38316/0 (socket says 172.21.15.118:38316) 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2affd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2d2c14bcd0 0x7f2d2c152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2b7fe700 1 -- 172.21.15.118:0/427262919 learned_addr learned my addr 172.21.15.118:0/427262919 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2a7fc700 1 --2- 172.21.15.118:0/427262919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c0bd580 0x7f2d2c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d29ffb700 1 -- 172.21.15.118:0/427262919 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682050533 0 0) 0x7f2d2c151f80 con 0x7f2d2c0bea90 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d29ffb700 1 -- 172.21.15.118:0/427262919 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d14003570 con 0x7f2d2c0bea90 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2affd700 1 -- 172.21.15.118:0/427262919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c0bd580 msgr2=0x7f2d2c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:03.800 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2affd700 1 --2- 172.21.15.118:0/427262919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c0bd580 0x7f2d2c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2affd700 1 -- 172.21.15.118:0/427262919 >> v1:172.21.15.118:6789/0 conn(0x7f2d2c0bea90 legacy=0x7f2d2c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2affd700 1 -- 172.21.15.118:0/427262919 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c152950 con 0x7f2d2c14bcd0 2024-08-18T13:52:03.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d2affd700 1 --2- 172.21.15.118:0/427262919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c14bcd0 0x7f2d2c152410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0098e0 tx=0x7f2d1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71bf7b43ddac749 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d29ffb700 1 -- 172.21.15.118:0/427262919 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c011630 con 0x7f2d2c14bcd0 2024-08-18T13:52:03.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d29ffb700 1 -- 172.21.15.118:0/427262919 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c011790 con 0x7f2d2c14bcd0 2024-08-18T13:52:03.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d29ffb700 1 -- 172.21.15.118:0/427262919 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c011b30 con 0x7f2d2c14bcd0 2024-08-18T13:52:03.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- 172.21.15.118:0/427262919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c14bcd0 msgr2=0x7f2d2c152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 --2- 172.21.15.118:0/427262919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c14bcd0 0x7f2d2c152410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0098e0 tx=0x7f2d1c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- 172.21.15.118:0/427262919 shutdown_connections 2024-08-18T13:52:03.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 --2- 172.21.15.118:0/427262919 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c0bd580 0x7f2d2c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 --2- 172.21.15.118:0/427262919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c14bcd0 0x7f2d2c152410 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- 172.21.15.118:0/427262919 >> 172.21.15.118:0/427262919 conn(0x7f2d2c0c7c40 msgr2=0x7f2d2c0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:03.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- 172.21.15.118:0/427262919 shutdown_connections 2024-08-18T13:52:03.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.794+0000 7f2d31814700 1 -- 172.21.15.118:0/427262919 wait complete. 2024-08-18T13:52:03.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 Processor -- start 2024-08-18T13:52:03.803 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 -- start start 2024-08-18T13:52:03.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c0bd580 0x7f2d2c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c0bea90 0x7f2d2c0c1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c14bcd0 0x7f2d2c0c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c145b50 con 0x7f2d2c0bea90 2024-08-18T13:52:03.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0c1180 con 0x7f2d2c0bd580 2024-08-18T13:52:03.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c14cc50 con 0x7f2d2c14bcd0 2024-08-18T13:52:03.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c0bea90 0x7f2d2c0c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c0bea90 0x7f2d2c0c1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:45208/0 (socket says 172.21.15.118:45208) 2024-08-18T13:52:03.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 -- 172.21.15.118:0/4092893869 learned_addr learned my addr 172.21.15.118:0/4092893869 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2affd700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c0bd580 0x7f2d2c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2b7fe700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c14bcd0 0x7f2d2c0c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 -- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c14bcd0 msgr2=0x7f2d2c0c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c14bcd0 0x7f2d2c0c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 -- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c0bd580 msgr2=0x7f2d2c0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c0bd580 0x7f2d2c0c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 -- 172.21.15.118:0/4092893869 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c009420 con 0x7f2d2c0bea90 2024-08-18T13:52:03.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2affd700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c0bd580 0x7f2d2c0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:03.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d2a7fc700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c0bea90 0x7f2d2c0c1610 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e930 tx=0x7f2d200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d12ffd700 1 -- 172.21.15.118:0/4092893869 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000c700 con 0x7f2d2c0bea90 2024-08-18T13:52:03.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d12ffd700 1 -- 172.21.15.118:0/4092893869 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2000c860 con 0x7f2d2c0bea90 2024-08-18T13:52:03.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c145d80 con 0x7f2d2c0bea90 2024-08-18T13:52:03.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d12ffd700 1 -- 172.21.15.118:0/4092893869 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000cc00 con 0x7f2d2c0bea90 2024-08-18T13:52:03.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.798+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c14d190 con 0x7f2d2c0bea90 2024-08-18T13:52:03.808 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.802+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c0c0f90 con 0x7f2d2c0bea90 2024-08-18T13:52:03.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.802+0000 7f2d12ffd700 1 -- 172.21.15.118:0/4092893869 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2d20013470 con 0x7f2d2c0bea90 2024-08-18T13:52:03.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.802+0000 7f2d12ffd700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d18060860 0x7f2d18062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:03.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.802+0000 7f2d12ffd700 1 -- 172.21.15.118:0/4092893869 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f2d2007db00 con 0x7f2d2c0bea90 2024-08-18T13:52:03.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.802+0000 7f2d2affd700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d18060860 0x7f2d18062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:03.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.802+0000 7f2d2affd700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d18060860 0x7f2d18062d10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0091e0 tx=0x7f2d1c017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:03.812 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.806+0000 7f2d12ffd700 1 -- 172.21.15.118:0/4092893869 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2d2004da10 con 0x7f2d2c0bea90 2024-08-18T13:52:03.975 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.970+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d2c0642f0 con 0x7f2d18060860 2024-08-18T13:52:03.994 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.990+0000 7f2d12ffd700 1 -- 172.21.15.118:0/4092893869 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370897 (secure 0 0 0) 0x7f2d2c0642f0 con 0x7f2d18060860 2024-08-18T13:52:03.999 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:52:04.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d18060860 msgr2=0x7f2d18062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:04.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d18060860 0x7f2d18062d10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0091e0 tx=0x7f2d1c017000 comp rx=0 tx=0).stop 2024-08-18T13:52:04.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c0bea90 msgr2=0x7f2d2c0c1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:04.002 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c0bea90 0x7f2d2c0c1610 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e930 tx=0x7f2d200091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 shutdown_connections 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2d18060860 0x7f2d18062d10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d2c14bcd0 0x7f2d2c0c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d2c0bea90 0x7f2d2c0c1610 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 --2- 172.21.15.118:0/4092893869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d2c0bd580 0x7f2d2c0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 >> 172.21.15.118:0/4092893869 conn(0x7f2d2c0c7c40 msgr2=0x7f2d2c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 shutdown_connections 2024-08-18T13:52:04.003 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:03.994+0000 7f2d31814700 1 -- 172.21.15.118:0/4092893869 wait complete. 2024-08-18T13:52:04.004 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:52:04.013 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-08-18T13:52:03.490257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1123626340,"num_objects":11047,"num_object_clones":0,"num_object_copies":43906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11047,"num_whiteouts":0,"num_read":22597,"num_read_kb":1390334,"num_write":28061,"num_write_kb":1250210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33755,"ondisk_log_size":33755,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4737576,"kb_used_data":2321320,"kb_used_omap":13,"kb_used_meta":2416178,"kb_avail":750237144,"statfs":{"total":773094113280,"available":768242835456,"internally_reserved":0,"allocated":2377031680,"data_stored":2374634208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2474166976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4899799,"num_objects":172,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":268,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004658"},"pg_stats":[{"pgid":"2.1f","version":"53'151","reported_seq":215,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:16.939395+0000","last_change":"2024-08-18T13:39:34.396129+0000","last_active":"2024-08-18T13:51:16.939395+0000","last_peered":"2024-08-18T13:51:16.939395+0000","last_clean":"2024-08-18T13:51:16.939395+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:51:16.939395+0000","last_undegraded":"2024-08-18T13:51:16.939395+0000","last_fullsized":"2024-08-18T13:51:16.939395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389178,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":28696,"num_write":150,"num_write_kb":8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766930+0000","last_change":"2024-08-18T13:43:55.555634+0000","last_active":"2024-08-18T13:51:51.766930+0000","last_peered":"2024-08-18T13:51:51.766930+0000","last_clean":"2024-08-18T13:51:51.766930+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:51:51.766930+0000","last_undegraded":"2024-08-18T13:51:51.766930+0000","last_fullsized":"2024-08-18T13:51:51.766930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:47:58.525363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064850389999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"54'676","reported_seq":3751,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.726737+0000","last_change":"2024-08-18T13:43:55.552107+0000","last_active":"2024-08-18T13:51:49.726737+0000","last_peered":"2024-08-18T13:51:49.726737+0000","last_clean":"2024-08-18T13:51:49.726737+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:51:49.726737+0000","last_undegraded":"2024-08-18T13:51:49.726737+0000","last_fullsized":"2024-08-18T13:51:49.726737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:43:39.435067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067303959999999996,"stat_sum":{"num_bytes":37566158,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":1013,"num_read_kb":36248,"num_write":673,"num_write_kb":41139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"54'78","reported_seq":126,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.670265+0000","last_change":"2024-08-18T13:39:34.391711+0000","last_active":"2024-08-18T13:50:55.670265+0000","last_peered":"2024-08-18T13:50:55.670265+0000","last_clean":"2024-08-18T13:50:55.670265+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:50:55.670265+0000","last_undegraded":"2024-08-18T13:50:55.670265+0000","last_fullsized":"2024-08-18T13:50:55.670265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":75,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"54'87","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:01.076425+0000","last_change":"2024-08-18T13:52:01.076369+0000","last_active":"2024-08-18T13:52:01.076425+0000","last_peered":"2024-08-18T13:52:01.076425+0000","last_clean":"2024-08-18T13:52:01.076425+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:52:01.076425+0000","last_undegraded":"2024-08-18T13:52:01.076425+0000","last_fullsized":"2024-08-18T13:52:01.076425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_clean_scrub_stamp":"2024-08-18T13:52:01.076152+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:46:50.073762+0000","scrub_duration":0.0039656070000000003,"objects_trimmed":0,"snaptrim_duration":0.0075430439999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"54'705","reported_seq":3560,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.599934+0000","last_change":"2024-08-18T13:43:55.555148+0000","last_active":"2024-08-18T13:51:49.599934+0000","last_peered":"2024-08-18T13:51:49.599934+0000","last_clean":"2024-08-18T13:51:49.599934+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:51:49.599934+0000","last_undegraded":"2024-08-18T13:51:49.599934+0000","last_fullsized":"2024-08-18T13:51:49.599934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075371290000000001,"stat_sum":{"num_bytes":29851183,"num_objects":370,"num_object_clones":0,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":792,"num_read_kb":20881,"num_write":708,"num_write_kb":37566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"54'194","reported_seq":259,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.297066+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:51:47.297066+0000","last_peered":"2024-08-18T13:51:47.297066+0000","last_clean":"2024-08-18T13:51:47.297066+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:51:47.297066+0000","last_undegraded":"2024-08-18T13:51:47.297066+0000","last_fullsized":"2024-08-18T13:51:47.297066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:20:25.461470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":69650,"num_write":192,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775007+0000","last_change":"2024-08-18T13:43:55.857350+0000","last_active":"2024-08-18T13:51:51.775007+0000","last_peered":"2024-08-18T13:51:51.775007+0000","last_clean":"2024-08-18T13:51:51.775007+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:51:51.775007+0000","last_undegraded":"2024-08-18T13:51:51.775007+0000","last_fullsized":"2024-08-18T13:51:51.775007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:41:41.700097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046085359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"54'680","reported_seq":3481,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.517007+0000","last_change":"2024-08-18T13:43:55.557895+0000","last_active":"2024-08-18T13:51:49.517007+0000","last_peered":"2024-08-18T13:51:49.517007+0000","last_clean":"2024-08-18T13:51:49.517007+0000","last_became_active":"2024-08-18T13:40:16.608743+0000","last_became_peered":"2024-08-18T13:40:16.608743+0000","last_unstale":"2024-08-18T13:51:49.517007+0000","last_undegraded":"2024-08-18T13:51:49.517007+0000","last_fullsized":"2024-08-18T13:51:49.517007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080275980000000004,"stat_sum":{"num_bytes":28774140,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":727,"num_read_kb":13856,"num_write":680,"num_write_kb":28426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"53'23","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.459383+0000","last_change":"2024-08-18T13:39:34.394647+0000","last_active":"2024-08-18T13:50:56.459383+0000","last_peered":"2024-08-18T13:50:56.459383+0000","last_clean":"2024-08-18T13:50:56.459383+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:50:56.459383+0000","last_undegraded":"2024-08-18T13:50:56.459383+0000","last_fullsized":"2024-08-18T13:50:56.459383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":25,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777014+0000","last_change":"2024-08-18T13:43:55.862914+0000","last_active":"2024-08-18T13:51:51.777014+0000","last_peered":"2024-08-18T13:51:51.777014+0000","last_clean":"2024-08-18T13:51:51.777014+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:51:51.777014+0000","last_undegraded":"2024-08-18T13:51:51.777014+0000","last_fullsized":"2024-08-18T13:51:51.777014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065299970000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"54'674","reported_seq":2903,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.468005+0000","last_change":"2024-08-18T13:43:55.555499+0000","last_active":"2024-08-18T13:51:49.468005+0000","last_peered":"2024-08-18T13:51:49.468005+0000","last_clean":"2024-08-18T13:51:49.468005+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:51:49.468005+0000","last_undegraded":"2024-08-18T13:51:49.468005+0000","last_fullsized":"2024-08-18T13:51:49.468005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078514629999999995,"stat_sum":{"num_bytes":11851705,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":166,"num_read_kb":1355,"num_write":673,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755374+0000","last_change":"2024-08-18T13:43:55.954106+0000","last_active":"2024-08-18T13:51:51.755374+0000","last_peered":"2024-08-18T13:51:51.755374+0000","last_clean":"2024-08-18T13:51:51.755374+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:51:51.755374+0000","last_undegraded":"2024-08-18T13:51:51.755374+0000","last_fullsized":"2024-08-18T13:51:51.755374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:34:46.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033699279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"54'228","reported_seq":287,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:58.748373+0000","last_change":"2024-08-18T13:39:34.396896+0000","last_active":"2024-08-18T13:51:58.748373+0000","last_peered":"2024-08-18T13:51:58.748373+0000","last_clean":"2024-08-18T13:51:58.748373+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:51:58.748373+0000","last_undegraded":"2024-08-18T13:51:58.748373+0000","last_fullsized":"2024-08-18T13:51:58.748373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:24:24.731607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12049255,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":23,"num_read_kb":45068,"num_write":223,"num_write_kb":11924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"54'638","reported_seq":3282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.659037+0000","last_change":"2024-08-18T13:43:55.558893+0000","last_active":"2024-08-18T13:51:49.659037+0000","last_peered":"2024-08-18T13:51:49.659037+0000","last_clean":"2024-08-18T13:51:49.659037+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:51:49.659037+0000","last_undegraded":"2024-08-18T13:51:49.659037+0000","last_fullsized":"2024-08-18T13:51:49.659037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059019540000000001,"stat_sum":{"num_bytes":16782769,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":581,"num_read_kb":14450,"num_write":639,"num_write_kb":24822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763887+0000","last_change":"2024-08-18T13:43:55.554204+0000","last_active":"2024-08-18T13:51:51.763887+0000","last_peered":"2024-08-18T13:51:51.763887+0000","last_clean":"2024-08-18T13:51:51.763887+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:51:51.763887+0000","last_undegraded":"2024-08-18T13:51:51.763887+0000","last_fullsized":"2024-08-18T13:51:51.763887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:20.785998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074252320000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"54'28","reported_seq":77,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.035340+0000","last_change":"2024-08-18T13:39:34.393908+0000","last_active":"2024-08-18T13:51:17.035340+0000","last_peered":"2024-08-18T13:51:17.035340+0000","last_clean":"2024-08-18T13:51:17.035340+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:51:17.035340+0000","last_undegraded":"2024-08-18T13:51:17.035340+0000","last_fullsized":"2024-08-18T13:51:17.035340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":28,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"54'709","reported_seq":3800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.742550+0000","last_change":"2024-08-18T13:43:55.558557+0000","last_active":"2024-08-18T13:51:49.742550+0000","last_peered":"2024-08-18T13:51:49.742550+0000","last_clean":"2024-08-18T13:51:49.742550+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:51:49.742550+0000","last_undegraded":"2024-08-18T13:51:49.742550+0000","last_fullsized":"2024-08-18T13:51:49.742550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057029610000000003,"stat_sum":{"num_bytes":33413248,"num_objects":347,"num_object_clones":0,"num_object_copies":1388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1028,"num_read_kb":27410,"num_write":707,"num_write_kb":41185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761963+0000","last_change":"2024-08-18T13:43:55.553624+0000","last_active":"2024-08-18T13:51:51.761963+0000","last_peered":"2024-08-18T13:51:51.761963+0000","last_clean":"2024-08-18T13:51:51.761963+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:51:51.761963+0000","last_undegraded":"2024-08-18T13:51:51.761963+0000","last_fullsized":"2024-08-18T13:51:51.761963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:18.383209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069243719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"53'13","reported_seq":58,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.426877+0000","last_change":"2024-08-18T13:39:34.389920+0000","last_active":"2024-08-18T13:50:57.426877+0000","last_peered":"2024-08-18T13:50:57.426877+0000","last_clean":"2024-08-18T13:50:57.426877+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:50:57.426877+0000","last_undegraded":"2024-08-18T13:50:57.426877+0000","last_fullsized":"2024-08-18T13:50:57.426877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":11,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'683","reported_seq":3571,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.731009+0000","last_change":"2024-08-18T13:43:56.142589+0000","last_active":"2024-08-18T13:51:49.731009+0000","last_peered":"2024-08-18T13:51:49.731009+0000","last_clean":"2024-08-18T13:51:49.731009+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:51:49.731009+0000","last_undegraded":"2024-08-18T13:51:49.731009+0000","last_fullsized":"2024-08-18T13:51:49.731009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079287170000000001,"stat_sum":{"num_bytes":20263748,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":825,"num_read_kb":25932,"num_write":679,"num_write_kb":28331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"54'92","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.756546+0000","last_change":"2024-08-18T13:43:55.558187+0000","last_active":"2024-08-18T13:51:51.756546+0000","last_peered":"2024-08-18T13:51:51.756546+0000","last_clean":"2024-08-18T13:51:51.756546+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:51:51.756546+0000","last_undegraded":"2024-08-18T13:51:51.756546+0000","last_fullsized":"2024-08-18T13:51:51.756546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068791520000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"54'106","reported_seq":153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.256673+0000","last_change":"2024-08-18T13:39:34.394379+0000","last_active":"2024-08-18T13:51:41.256673+0000","last_peered":"2024-08-18T13:51:41.256673+0000","last_clean":"2024-08-18T13:51:41.256673+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:51:41.256673+0000","last_undegraded":"2024-08-18T13:51:41.256673+0000","last_fullsized":"2024-08-18T13:51:41.256673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:34.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":108,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"54'617","reported_seq":3453,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.737566+0000","last_change":"2024-08-18T13:43:55.559559+0000","last_active":"2024-08-18T13:51:49.737566+0000","last_peered":"2024-08-18T13:51:49.737566+0000","last_clean":"2024-08-18T13:51:49.737566+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:51:49.737566+0000","last_undegraded":"2024-08-18T13:51:49.737566+0000","last_fullsized":"2024-08-18T13:51:49.737566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004078028,"stat_sum":{"num_bytes":24272349,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":773,"num_read_kb":25488,"num_write":617,"num_write_kb":28093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'93","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762526+0000","last_change":"2024-08-18T13:43:56.154543+0000","last_active":"2024-08-18T13:51:51.762526+0000","last_peered":"2024-08-18T13:51:51.762526+0000","last_clean":"2024-08-18T13:51:51.762526+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:51:51.762526+0000","last_undegraded":"2024-08-18T13:51:51.762526+0000","last_fullsized":"2024-08-18T13:51:51.762526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060661889999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"54'111","reported_seq":160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.305665+0000","last_change":"2024-08-18T13:39:34.395587+0000","last_active":"2024-08-18T13:51:17.305665+0000","last_peered":"2024-08-18T13:51:17.305665+0000","last_clean":"2024-08-18T13:51:17.305665+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:51:17.305665+0000","last_undegraded":"2024-08-18T13:51:17.305665+0000","last_fullsized":"2024-08-18T13:51:17.305665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:17:59.639933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":111,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"54'720","reported_seq":2970,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.475489+0000","last_change":"2024-08-18T13:43:55.557702+0000","last_active":"2024-08-18T13:51:49.475489+0000","last_peered":"2024-08-18T13:51:49.475489+0000","last_clean":"2024-08-18T13:51:49.475489+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:51:49.475489+0000","last_undegraded":"2024-08-18T13:51:49.475489+0000","last_fullsized":"2024-08-18T13:51:49.475489+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079796720000000002,"stat_sum":{"num_bytes":17871290,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":187,"num_read_kb":7768,"num_write":717,"num_write_kb":21887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781486+0000","last_change":"2024-08-18T13:43:55.555491+0000","last_active":"2024-08-18T13:51:51.781486+0000","last_peered":"2024-08-18T13:51:51.781486+0000","last_clean":"2024-08-18T13:51:51.781486+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:51:51.781486+0000","last_undegraded":"2024-08-18T13:51:51.781486+0000","last_fullsized":"2024-08-18T13:51:51.781486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054340660000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"54'66","reported_seq":117,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:53.746477+0000","last_change":"2024-08-18T13:39:34.389033+0000","last_active":"2024-08-18T13:51:53.746477+0000","last_peered":"2024-08-18T13:51:53.746477+0000","last_clean":"2024-08-18T13:51:53.746477+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:51:53.746477+0000","last_undegraded":"2024-08-18T13:51:53.746477+0000","last_fullsized":"2024-08-18T13:51:53.746477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":184,"num_write":164,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"54'714","reported_seq":2929,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.710846+0000","last_change":"2024-08-18T13:43:55.550748+0000","last_active":"2024-08-18T13:51:49.710846+0000","last_peered":"2024-08-18T13:51:49.710846+0000","last_clean":"2024-08-18T13:51:49.710846+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:51:49.710846+0000","last_undegraded":"2024-08-18T13:51:49.710846+0000","last_fullsized":"2024-08-18T13:51:49.710846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:51:52.061177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057306029999999999,"stat_sum":{"num_bytes":13242151,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":153,"num_read_kb":5341,"num_write":712,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'90","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758695+0000","last_change":"2024-08-18T13:43:56.143836+0000","last_active":"2024-08-18T13:51:51.758695+0000","last_peered":"2024-08-18T13:51:51.758695+0000","last_clean":"2024-08-18T13:51:51.758695+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:51:51.758695+0000","last_undegraded":"2024-08-18T13:51:51.758695+0000","last_fullsized":"2024-08-18T13:51:51.758695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064599770000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"54'289","reported_seq":337,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:33.863944+0000","last_change":"2024-08-18T13:39:34.397155+0000","last_active":"2024-08-18T13:51:33.863944+0000","last_peered":"2024-08-18T13:51:33.863944+0000","last_clean":"2024-08-18T13:51:33.863944+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:51:33.863944+0000","last_undegraded":"2024-08-18T13:51:33.863944+0000","last_fullsized":"2024-08-18T13:51:33.863944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:51.749882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":12176,"num_write":284,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"54'704","reported_seq":3426,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.674395+0000","last_change":"2024-08-18T13:43:56.151855+0000","last_active":"2024-08-18T13:51:49.674395+0000","last_peered":"2024-08-18T13:51:49.674395+0000","last_clean":"2024-08-18T13:51:49.674395+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:51:49.674395+0000","last_undegraded":"2024-08-18T13:51:49.674395+0000","last_fullsized":"2024-08-18T13:51:49.674395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060004689999999996,"stat_sum":{"num_bytes":28446730,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":659,"num_read_kb":21792,"num_write":702,"num_write_kb":28155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"54'84","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775443+0000","last_change":"2024-08-18T13:43:55.863531+0000","last_active":"2024-08-18T13:51:51.775443+0000","last_peered":"2024-08-18T13:51:51.775443+0000","last_clean":"2024-08-18T13:51:51.775443+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:51:51.775443+0000","last_undegraded":"2024-08-18T13:51:51.775443+0000","last_fullsized":"2024-08-18T13:51:51.775443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006455484,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"54'657","reported_seq":701,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.995066+0000","last_change":"2024-08-18T13:39:34.380173+0000","last_active":"2024-08-18T13:51:38.995066+0000","last_peered":"2024-08-18T13:51:38.995066+0000","last_clean":"2024-08-18T13:51:38.995066+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:51:38.995066+0000","last_undegraded":"2024-08-18T13:51:38.995066+0000","last_fullsized":"2024-08-18T13:51:38.995066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"54'691","reported_seq":3344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.584569+0000","last_change":"2024-08-18T13:43:56.141004+0000","last_active":"2024-08-18T13:51:49.584569+0000","last_peered":"2024-08-18T13:51:49.584569+0000","last_clean":"2024-08-18T13:51:49.584569+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:51:49.584569+0000","last_undegraded":"2024-08-18T13:51:49.584569+0000","last_fullsized":"2024-08-18T13:51:49.584569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067583870000000002,"stat_sum":{"num_bytes":38683931,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":590,"num_read_kb":16518,"num_write":694,"num_write_kb":42167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"54'97","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758832+0000","last_change":"2024-08-18T13:43:55.559256+0000","last_active":"2024-08-18T13:51:51.758832+0000","last_peered":"2024-08-18T13:51:51.758832+0000","last_clean":"2024-08-18T13:51:51.758832+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:51:51.758832+0000","last_undegraded":"2024-08-18T13:51:51.758832+0000","last_fullsized":"2024-08-18T13:51:51.758832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005854756,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"53'383","reported_seq":438,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:55.690458+0000","last_change":"2024-08-18T13:39:34.382845+0000","last_active":"2024-08-18T13:50:55.690458+0000","last_peered":"2024-08-18T13:50:55.690458+0000","last_clean":"2024-08-18T13:50:55.690458+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:50:55.690458+0000","last_undegraded":"2024-08-18T13:50:55.690458+0000","last_fullsized":"2024-08-18T13:50:55.690458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:53:17.430090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":43191,"num_write":382,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"54'661","reported_seq":4461,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.604795+0000","last_change":"2024-08-18T13:43:56.144223+0000","last_active":"2024-08-18T13:51:49.604795+0000","last_peered":"2024-08-18T13:51:49.604795+0000","last_clean":"2024-08-18T13:51:49.604795+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:51:49.604795+0000","last_undegraded":"2024-08-18T13:51:49.604795+0000","last_fullsized":"2024-08-18T13:51:49.604795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067969390000000001,"stat_sum":{"num_bytes":51061167,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":1737,"num_read_kb":43040,"num_write":661,"num_write_kb":53481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"54'83","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758647+0000","last_change":"2024-08-18T13:43:56.143663+0000","last_active":"2024-08-18T13:51:51.758647+0000","last_peered":"2024-08-18T13:51:51.758647+0000","last_clean":"2024-08-18T13:51:51.758647+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:51:51.758647+0000","last_undegraded":"2024-08-18T13:51:51.758647+0000","last_fullsized":"2024-08-18T13:51:51.758647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063915389999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"53'120","reported_seq":283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.353455+0000","last_change":"2024-08-18T13:39:34.382821+0000","last_active":"2024-08-18T13:50:51.353455+0000","last_peered":"2024-08-18T13:50:51.353455+0000","last_clean":"2024-08-18T13:50:51.353455+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:50:51.353455+0000","last_undegraded":"2024-08-18T13:50:51.353455+0000","last_fullsized":"2024-08-18T13:50:51.353455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":638,"num_write":117,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"54'643","reported_seq":3598,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.506760+0000","last_change":"2024-08-18T13:43:56.143097+0000","last_active":"2024-08-18T13:51:49.506760+0000","last_peered":"2024-08-18T13:51:49.506760+0000","last_clean":"2024-08-18T13:51:49.506760+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:51:49.506760+0000","last_undegraded":"2024-08-18T13:51:49.506760+0000","last_fullsized":"2024-08-18T13:51:49.506760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079242040000000007,"stat_sum":{"num_bytes":41790406,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":892,"num_read_kb":23319,"num_write":639,"num_write_kb":45248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'82","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762541+0000","last_change":"2024-08-18T13:43:56.149372+0000","last_active":"2024-08-18T13:51:51.762541+0000","last_peered":"2024-08-18T13:51:51.762541+0000","last_clean":"2024-08-18T13:51:51.762541+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:51:51.762541+0000","last_undegraded":"2024-08-18T13:51:51.762541+0000","last_fullsized":"2024-08-18T13:51:51.762541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041495459999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"54'234","reported_seq":290,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:57.626190+0000","last_change":"2024-08-18T13:39:34.396959+0000","last_active":"2024-08-18T13:50:57.626190+0000","last_peered":"2024-08-18T13:50:57.626190+0000","last_clean":"2024-08-18T13:50:57.626190+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:50:57.626190+0000","last_undegraded":"2024-08-18T13:50:57.626190+0000","last_fullsized":"2024-08-18T13:50:57.626190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:24.239026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":49152,"num_write":231,"num_write_kb":12421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"54'716","reported_seq":3217,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.657979+0000","last_change":"2024-08-18T13:43:56.153929+0000","last_active":"2024-08-18T13:51:49.657979+0000","last_peered":"2024-08-18T13:51:49.657979+0000","last_clean":"2024-08-18T13:51:49.657979+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:51:49.657979+0000","last_undegraded":"2024-08-18T13:51:49.657979+0000","last_fullsized":"2024-08-18T13:51:49.657979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058974379999999996,"stat_sum":{"num_bytes":30462084,"num_objects":352,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":439,"num_read_kb":15189,"num_write":710,"num_write_kb":30067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779966+0000","last_change":"2024-08-18T13:43:55.552534+0000","last_active":"2024-08-18T13:51:51.779966+0000","last_peered":"2024-08-18T13:51:51.779966+0000","last_clean":"2024-08-18T13:51:51.779966+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:51:51.779966+0000","last_undegraded":"2024-08-18T13:51:51.779966+0000","last_fullsized":"2024-08-18T13:51:51.779966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068182370000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"54'982","reported_seq":1027,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.365615+0000","last_change":"2024-08-18T13:39:34.388132+0000","last_active":"2024-08-18T13:51:17.365615+0000","last_peered":"2024-08-18T13:51:17.365615+0000","last_clean":"2024-08-18T13:51:17.365615+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:51:17.365615+0000","last_undegraded":"2024-08-18T13:51:17.365615+0000","last_fullsized":"2024-08-18T13:51:17.365615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2087,"num_write":979,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:49:30.943967+0000","last_change":"2024-08-18T13:39:22.782727+0000","last_active":"2024-08-18T13:49:30.943967+0000","last_peered":"2024-08-18T13:49:30.943967+0000","last_clean":"2024-08-18T13:49:30.943967+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:49:30.943967+0000","last_undegraded":"2024-08-18T13:49:30.943967+0000","last_fullsized":"2024-08-18T13:49:30.943967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:21.399940+0000","last_clean_scrub_stamp":"2024-08-18T13:39:21.399940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:21.308336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"54'664","reported_seq":3524,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.497597+0000","last_change":"2024-08-18T13:43:55.956673+0000","last_active":"2024-08-18T13:51:49.497597+0000","last_peered":"2024-08-18T13:51:49.497597+0000","last_clean":"2024-08-18T13:51:49.497597+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:51:49.497597+0000","last_undegraded":"2024-08-18T13:51:49.497597+0000","last_fullsized":"2024-08-18T13:51:49.497597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:31:22.337449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035684480000000001,"stat_sum":{"num_bytes":39930170,"num_objects":364,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":798,"num_read_kb":26403,"num_write":662,"num_write_kb":47533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779241+0000","last_change":"2024-08-18T13:43:55.552313+0000","last_active":"2024-08-18T13:51:51.779241+0000","last_peered":"2024-08-18T13:51:51.779241+0000","last_clean":"2024-08-18T13:51:51.779241+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:51:51.779241+0000","last_undegraded":"2024-08-18T13:51:51.779241+0000","last_fullsized":"2024-08-18T13:51:51.779241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:56.896257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057711639999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"54'76","reported_seq":118,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.291821+0000","last_change":"2024-08-18T13:39:34.382868+0000","last_active":"2024-08-18T13:51:47.291821+0000","last_peered":"2024-08-18T13:51:47.291821+0000","last_clean":"2024-08-18T13:51:47.291821+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:51:47.291821+0000","last_undegraded":"2024-08-18T13:51:47.291821+0000","last_fullsized":"2024-08-18T13:51:47.291821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:23.072140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"54'688","reported_seq":4127,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.578882+0000","last_change":"2024-08-18T13:43:55.953251+0000","last_active":"2024-08-18T13:51:49.578882+0000","last_peered":"2024-08-18T13:51:49.578882+0000","last_clean":"2024-08-18T13:51:49.578882+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:51:49.578882+0000","last_undegraded":"2024-08-18T13:51:49.578882+0000","last_fullsized":"2024-08-18T13:51:49.578882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:35:47.170881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032433420000000002,"stat_sum":{"num_bytes":54557211,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1377,"num_read_kb":50055,"num_write":687,"num_write_kb":57734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"54'95","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.756325+0000","last_change":"2024-08-18T13:43:55.918416+0000","last_active":"2024-08-18T13:51:51.756325+0000","last_peered":"2024-08-18T13:51:51.756325+0000","last_clean":"2024-08-18T13:51:51.756325+0000","last_became_active":"2024-08-18T13:40:16.614814+0000","last_became_peered":"2024-08-18T13:40:16.614814+0000","last_unstale":"2024-08-18T13:51:51.756325+0000","last_undegraded":"2024-08-18T13:51:51.756325+0000","last_fullsized":"2024-08-18T13:51:51.756325+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:02:39.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033975450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'31","reported_seq":86,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:47.298035+0000","last_change":"2024-08-18T13:39:34.384915+0000","last_active":"2024-08-18T13:51:47.298035+0000","last_peered":"2024-08-18T13:51:47.298035+0000","last_clean":"2024-08-18T13:51:47.298035+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:51:47.298035+0000","last_undegraded":"2024-08-18T13:51:47.298035+0000","last_fullsized":"2024-08-18T13:51:47.298035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":33,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"54'670","reported_seq":3235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.683385+0000","last_change":"2024-08-18T13:43:56.153281+0000","last_active":"2024-08-18T13:51:49.683385+0000","last_peered":"2024-08-18T13:51:49.683385+0000","last_clean":"2024-08-18T13:51:49.683385+0000","last_became_active":"2024-08-18T13:40:16.609908+0000","last_became_peered":"2024-08-18T13:40:16.609908+0000","last_unstale":"2024-08-18T13:51:49.683385+0000","last_undegraded":"2024-08-18T13:51:49.683385+0000","last_fullsized":"2024-08-18T13:51:49.683385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069871079999999997,"stat_sum":{"num_bytes":26719276,"num_objects":358,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":491,"num_read_kb":18148,"num_write":671,"num_write_kb":26414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4,5],"acting":[0,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'87","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784591+0000","last_change":"2024-08-18T13:43:55.553120+0000","last_active":"2024-08-18T13:51:51.784591+0000","last_peered":"2024-08-18T13:51:51.784591+0000","last_clean":"2024-08-18T13:51:51.784591+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:51:51.784591+0000","last_undegraded":"2024-08-18T13:51:51.784591+0000","last_fullsized":"2024-08-18T13:51:51.784591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:22:28.476139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057951929999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"53'29","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.394499+0000","last_change":"2024-08-18T13:39:34.387182+0000","last_active":"2024-08-18T13:50:56.394499+0000","last_peered":"2024-08-18T13:50:56.394499+0000","last_clean":"2024-08-18T13:50:56.394499+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:50:56.394499+0000","last_undegraded":"2024-08-18T13:50:56.394499+0000","last_fullsized":"2024-08-18T13:50:56.394499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:02.825398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":30,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"54'688","reported_seq":2818,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.687818+0000","last_change":"2024-08-18T13:43:55.859666+0000","last_active":"2024-08-18T13:51:49.687818+0000","last_peered":"2024-08-18T13:51:49.687818+0000","last_clean":"2024-08-18T13:51:49.687818+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:51:49.687818+0000","last_undegraded":"2024-08-18T13:51:49.687818+0000","last_fullsized":"2024-08-18T13:51:49.687818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:46:55.410810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070733389999999997,"stat_sum":{"num_bytes":17282460,"num_objects":346,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":68,"num_read_kb":8224,"num_write":686,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"54'66","reported_seq":2129,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783626+0000","last_change":"2024-08-18T13:43:55.551809+0000","last_active":"2024-08-18T13:51:51.783626+0000","last_peered":"2024-08-18T13:51:51.783626+0000","last_clean":"2024-08-18T13:51:51.783626+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:51:51.783626+0000","last_undegraded":"2024-08-18T13:51:51.783626+0000","last_fullsized":"2024-08-18T13:51:51.783626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:00:34.193845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060906930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"53'26","reported_seq":81,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.603056+0000","last_change":"2024-08-18T13:39:34.387707+0000","last_active":"2024-08-18T13:50:56.603056+0000","last_peered":"2024-08-18T13:50:56.603056+0000","last_clean":"2024-08-18T13:50:56.603056+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:50:56.603056+0000","last_undegraded":"2024-08-18T13:50:56.603056+0000","last_fullsized":"2024-08-18T13:50:56.603056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":27,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"54'695","reported_seq":3631,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.700276+0000","last_change":"2024-08-18T13:43:55.557389+0000","last_active":"2024-08-18T13:51:49.700276+0000","last_peered":"2024-08-18T13:51:49.700276+0000","last_clean":"2024-08-18T13:51:49.700276+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:51:49.700276+0000","last_undegraded":"2024-08-18T13:51:49.700276+0000","last_fullsized":"2024-08-18T13:51:49.700276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011606655,"stat_sum":{"num_bytes":44119789,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":873,"num_read_kb":37157,"num_write":696,"num_write_kb":47501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"54'91","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784041+0000","last_change":"2024-08-18T13:43:55.553915+0000","last_active":"2024-08-18T13:51:51.784041+0000","last_peered":"2024-08-18T13:51:51.784041+0000","last_clean":"2024-08-18T13:51:51.784041+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:51:51.784041+0000","last_undegraded":"2024-08-18T13:51:51.784041+0000","last_fullsized":"2024-08-18T13:51:51.784041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:59:08.640018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053019490000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"53'1204","reported_seq":1276,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.252643+0000","last_change":"2024-08-18T13:39:34.378468+0000","last_active":"2024-08-18T13:51:17.252643+0000","last_peered":"2024-08-18T13:51:17.252643+0000","last_clean":"2024-08-18T13:51:17.252643+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:51:17.252643+0000","last_undegraded":"2024-08-18T13:51:17.252643+0000","last_fullsized":"2024-08-18T13:51:17.252643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":78,"num_read_kb":12399,"num_write":1202,"num_write_kb":8830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'657","reported_seq":3517,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.698516+0000","last_change":"2024-08-18T13:43:56.154837+0000","last_active":"2024-08-18T13:51:49.698516+0000","last_peered":"2024-08-18T13:51:49.698516+0000","last_clean":"2024-08-18T13:51:49.698516+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:51:49.698516+0000","last_undegraded":"2024-08-18T13:51:49.698516+0000","last_fullsized":"2024-08-18T13:51:49.698516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054104280000000001,"stat_sum":{"num_bytes":33280018,"num_objects":296,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296,"num_whiteouts":0,"num_read":797,"num_read_kb":22024,"num_write":655,"num_write_kb":32783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757105+0000","last_change":"2024-08-18T13:43:55.959459+0000","last_active":"2024-08-18T13:51:51.757105+0000","last_peered":"2024-08-18T13:51:51.757105+0000","last_clean":"2024-08-18T13:51:51.757105+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:51:51.757105+0000","last_undegraded":"2024-08-18T13:51:51.757105+0000","last_fullsized":"2024-08-18T13:51:51.757105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044255090000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"54'5","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.846481+0000","last_change":"2024-08-18T13:39:34.385437+0000","last_active":"2024-08-18T13:50:51.846481+0000","last_peered":"2024-08-18T13:50:51.846481+0000","last_clean":"2024-08-18T13:50:51.846481+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:50:51.846481+0000","last_undegraded":"2024-08-18T13:50:51.846481+0000","last_fullsized":"2024-08-18T13:50:51.846481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:14.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"54'660","reported_seq":2779,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.587481+0000","last_change":"2024-08-18T13:43:55.555242+0000","last_active":"2024-08-18T13:51:49.587481+0000","last_peered":"2024-08-18T13:51:49.587481+0000","last_clean":"2024-08-18T13:51:49.587481+0000","last_became_active":"2024-08-18T13:40:16.915369+0000","last_became_peered":"2024-08-18T13:40:16.915369+0000","last_unstale":"2024-08-18T13:51:49.587481+0000","last_undegraded":"2024-08-18T13:51:49.587481+0000","last_fullsized":"2024-08-18T13:51:49.587481+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:50:23.633304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005619059,"stat_sum":{"num_bytes":16977484,"num_objects":353,"num_object_clones":0,"num_object_copies":1412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":46,"num_read_kb":4277,"num_write":660,"num_write_kb":20975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,6],"acting":[3,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"54'101","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784667+0000","last_change":"2024-08-18T13:43:55.551514+0000","last_active":"2024-08-18T13:51:51.784667+0000","last_peered":"2024-08-18T13:51:51.784667+0000","last_clean":"2024-08-18T13:51:51.784667+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:51:51.784667+0000","last_undegraded":"2024-08-18T13:51:51.784667+0000","last_fullsized":"2024-08-18T13:51:51.784667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:51:16.928920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064019180000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"53'2","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-08-18T13:50:28.547615+0000","last_change":"2024-08-18T13:39:34.393977+0000","last_active":"2024-08-18T13:50:28.547615+0000","last_peered":"2024-08-18T13:50:28.547615+0000","last_clean":"2024-08-18T13:50:28.547615+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:50:28.547615+0000","last_undegraded":"2024-08-18T13:50:28.547615+0000","last_fullsized":"2024-08-18T13:50:28.547615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:43.720510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"54'625","reported_seq":3598,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.686816+0000","last_change":"2024-08-18T13:43:55.554811+0000","last_active":"2024-08-18T13:51:49.686816+0000","last_peered":"2024-08-18T13:51:49.686816+0000","last_clean":"2024-08-18T13:51:49.686816+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:51:49.686816+0000","last_undegraded":"2024-08-18T13:51:49.686816+0000","last_fullsized":"2024-08-18T13:51:49.686816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:42:33.163161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074125900000000002,"stat_sum":{"num_bytes":40693231,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":911,"num_read_kb":37155,"num_write":622,"num_write_kb":48255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'85","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778687+0000","last_change":"2024-08-18T13:43:56.154386+0000","last_active":"2024-08-18T13:51:51.778687+0000","last_peered":"2024-08-18T13:51:51.778687+0000","last_clean":"2024-08-18T13:51:51.778687+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:51:51.778687+0000","last_undegraded":"2024-08-18T13:51:51.778687+0000","last_fullsized":"2024-08-18T13:51:51.778687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059765870000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"54'216","reported_seq":284,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.583930+0000","last_change":"2024-08-18T13:39:34.378915+0000","last_active":"2024-08-18T13:51:41.583930+0000","last_peered":"2024-08-18T13:51:41.583930+0000","last_clean":"2024-08-18T13:51:41.583930+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:51:41.583930+0000","last_undegraded":"2024-08-18T13:51:41.583930+0000","last_fullsized":"2024-08-18T13:51:41.583930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":45098,"num_write":215,"num_write_kb":8331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'629","reported_seq":3941,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.453295+0000","last_change":"2024-08-18T13:43:55.862627+0000","last_active":"2024-08-18T13:51:49.453295+0000","last_peered":"2024-08-18T13:51:49.453295+0000","last_clean":"2024-08-18T13:51:49.453295+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:51:49.453295+0000","last_undegraded":"2024-08-18T13:51:49.453295+0000","last_fullsized":"2024-08-18T13:51:49.453295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:05:00.614003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084356780000000003,"stat_sum":{"num_bytes":50488256,"num_objects":306,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":1250,"num_read_kb":43248,"num_write":629,"num_write_kb":49603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'84","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779153+0000","last_change":"2024-08-18T13:43:55.551442+0000","last_active":"2024-08-18T13:51:51.779153+0000","last_peered":"2024-08-18T13:51:51.779153+0000","last_clean":"2024-08-18T13:51:51.779153+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:51:51.779153+0000","last_undegraded":"2024-08-18T13:51:51.779153+0000","last_fullsized":"2024-08-18T13:51:51.779153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:01:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065977270000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"54'233","reported_seq":306,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:43.734491+0000","last_change":"2024-08-18T13:39:34.394961+0000","last_active":"2024-08-18T13:51:43.734491+0000","last_peered":"2024-08-18T13:51:43.734491+0000","last_clean":"2024-08-18T13:51:43.734491+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:51:43.734491+0000","last_undegraded":"2024-08-18T13:51:43.734491+0000","last_fullsized":"2024-08-18T13:51:43.734491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:57:02.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583572,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":53278,"num_write":226,"num_write_kb":12432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"54'671","reported_seq":3263,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.572954+0000","last_change":"2024-08-18T13:43:56.142175+0000","last_active":"2024-08-18T13:51:49.572954+0000","last_peered":"2024-08-18T13:51:49.572954+0000","last_clean":"2024-08-18T13:51:49.572954+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:51:49.572954+0000","last_undegraded":"2024-08-18T13:51:49.572954+0000","last_fullsized":"2024-08-18T13:51:49.572954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076714670000000004,"stat_sum":{"num_bytes":16086174,"num_objects":343,"num_object_clones":0,"num_object_copies":1372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":529,"num_read_kb":12373,"num_write":668,"num_write_kb":24206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"54'86","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783578+0000","last_change":"2024-08-18T13:43:55.559262+0000","last_active":"2024-08-18T13:51:51.783578+0000","last_peered":"2024-08-18T13:51:51.783578+0000","last_clean":"2024-08-18T13:51:51.783578+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:51:51.783578+0000","last_undegraded":"2024-08-18T13:51:51.783578+0000","last_fullsized":"2024-08-18T13:51:51.783578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039212459999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"53'169","reported_seq":228,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:17.287022+0000","last_change":"2024-08-18T13:39:34.386516+0000","last_active":"2024-08-18T13:51:17.287022+0000","last_peered":"2024-08-18T13:51:17.287022+0000","last_clean":"2024-08-18T13:51:17.287022+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:51:17.287022+0000","last_undegraded":"2024-08-18T13:51:17.287022+0000","last_fullsized":"2024-08-18T13:51:17.287022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8216,"num_write":168,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'644","reported_seq":3279,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.705949+0000","last_change":"2024-08-18T13:43:55.559022+0000","last_active":"2024-08-18T13:51:49.705949+0000","last_peered":"2024-08-18T13:51:49.705949+0000","last_clean":"2024-08-18T13:51:49.705949+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:51:49.705949+0000","last_undegraded":"2024-08-18T13:51:49.705949+0000","last_fullsized":"2024-08-18T13:51:49.705949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059375670000000004,"stat_sum":{"num_bytes":15740586,"num_objects":325,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":572,"num_read_kb":18359,"num_write":643,"num_write_kb":23895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"54'89","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778091+0000","last_change":"2024-08-18T13:43:56.153537+0000","last_active":"2024-08-18T13:51:51.778091+0000","last_peered":"2024-08-18T13:51:51.778091+0000","last_clean":"2024-08-18T13:51:51.778091+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:51:51.778091+0000","last_undegraded":"2024-08-18T13:51:51.778091+0000","last_fullsized":"2024-08-18T13:51:51.778091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057226309999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"54'233","reported_seq":282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:56.891732+0000","last_change":"2024-08-18T13:39:34.391504+0000","last_active":"2024-08-18T13:50:56.891732+0000","last_peered":"2024-08-18T13:50:56.891732+0000","last_clean":"2024-08-18T13:50:56.891732+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:50:56.891732+0000","last_undegraded":"2024-08-18T13:50:56.891732+0000","last_fullsized":"2024-08-18T13:50:56.891732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:22:24.374092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":20397,"num_write":232,"num_write_kb":4254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"54'677","reported_seq":3169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.442883+0000","last_change":"2024-08-18T13:43:55.553346+0000","last_active":"2024-08-18T13:51:49.442883+0000","last_peered":"2024-08-18T13:51:49.442883+0000","last_clean":"2024-08-18T13:51:49.442883+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:51:49.442883+0000","last_undegraded":"2024-08-18T13:51:49.442883+0000","last_fullsized":"2024-08-18T13:51:49.442883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:04:34.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073620359999999998,"stat_sum":{"num_bytes":31240743,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":431,"num_read_kb":13577,"num_write":670,"num_write_kb":34133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"54'106","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778915+0000","last_change":"2024-08-18T13:43:55.861465+0000","last_active":"2024-08-18T13:51:51.778915+0000","last_peered":"2024-08-18T13:51:51.778915+0000","last_clean":"2024-08-18T13:51:51.778915+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:51:51.778915+0000","last_undegraded":"2024-08-18T13:51:51.778915+0000","last_fullsized":"2024-08-18T13:51:51.778915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:21.290442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059613369999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"54'9","reported_seq":51,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:41.581557+0000","last_change":"2024-08-18T13:39:34.389988+0000","last_active":"2024-08-18T13:51:41.581557+0000","last_peered":"2024-08-18T13:51:41.581557+0000","last_clean":"2024-08-18T13:51:41.581557+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:51:41.581557+0000","last_undegraded":"2024-08-18T13:51:41.581557+0000","last_fullsized":"2024-08-18T13:51:41.581557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:54.128522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"54'699","reported_seq":3480,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.480893+0000","last_change":"2024-08-18T13:43:55.553902+0000","last_active":"2024-08-18T13:51:49.480893+0000","last_peered":"2024-08-18T13:51:49.480893+0000","last_clean":"2024-08-18T13:51:49.480893+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:51:49.480893+0000","last_undegraded":"2024-08-18T13:51:49.480893+0000","last_fullsized":"2024-08-18T13:51:49.480893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:52:03.601499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076808309999999999,"stat_sum":{"num_bytes":29636158,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":719,"num_read_kb":24704,"num_write":698,"num_write_kb":38783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"54'650","reported_seq":3293,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.152118+0000","last_change":"2024-08-18T13:43:56.152924+0000","last_active":"2024-08-18T13:51:49.152118+0000","last_peered":"2024-08-18T13:51:49.152118+0000","last_clean":"2024-08-18T13:51:49.152118+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:51:49.152118+0000","last_undegraded":"2024-08-18T13:51:49.152118+0000","last_fullsized":"2024-08-18T13:51:49.152118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068864499999999997,"stat_sum":{"num_bytes":24960867,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":580,"num_read_kb":20952,"num_write":650,"num_write_kb":24715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"54'91","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761812+0000","last_change":"2024-08-18T13:43:56.139883+0000","last_active":"2024-08-18T13:51:51.761812+0000","last_peered":"2024-08-18T13:51:51.761812+0000","last_clean":"2024-08-18T13:51:51.761812+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:51:51.761812+0000","last_undegraded":"2024-08-18T13:51:51.761812+0000","last_fullsized":"2024-08-18T13:51:51.761812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066095759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"54'199","reported_seq":252,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:42.157051+0000","last_change":"2024-08-18T13:39:34.387963+0000","last_active":"2024-08-18T13:51:42.157051+0000","last_peered":"2024-08-18T13:51:42.157051+0000","last_clean":"2024-08-18T13:51:42.157051+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:51:42.157051+0000","last_undegraded":"2024-08-18T13:51:42.157051+0000","last_fullsized":"2024-08-18T13:51:42.157051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:54:50.333442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":197,"num_write_kb":12441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"54'658","reported_seq":3187,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.395492+0000","last_change":"2024-08-18T13:43:55.958984+0000","last_active":"2024-08-18T13:51:49.395492+0000","last_peered":"2024-08-18T13:51:49.395492+0000","last_clean":"2024-08-18T13:51:49.395492+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:51:49.395492+0000","last_undegraded":"2024-08-18T13:51:49.395492+0000","last_fullsized":"2024-08-18T13:51:49.395492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:14:00.947192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047613839999999996,"stat_sum":{"num_bytes":20867535,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":467,"num_read_kb":16390,"num_write":656,"num_write_kb":20726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764935+0000","last_change":"2024-08-18T13:43:55.552907+0000","last_active":"2024-08-18T13:51:51.764935+0000","last_peered":"2024-08-18T13:51:51.764935+0000","last_clean":"2024-08-18T13:51:51.764935+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:51:51.764935+0000","last_undegraded":"2024-08-18T13:51:51.764935+0000","last_fullsized":"2024-08-18T13:51:51.764935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:27.568429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068339799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"53'103","reported_seq":150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:50:51.907236+0000","last_change":"2024-08-18T13:39:34.391684+0000","last_active":"2024-08-18T13:50:51.907236+0000","last_peered":"2024-08-18T13:50:51.907236+0000","last_clean":"2024-08-18T13:50:51.907236+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:50:51.907236+0000","last_undegraded":"2024-08-18T13:50:51.907236+0000","last_fullsized":"2024-08-18T13:50:51.907236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:48:06.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":103,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"54'131","reported_seq":186,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:34.672086+0000","last_change":"2024-08-18T13:39:34.385626+0000","last_active":"2024-08-18T13:51:34.672086+0000","last_peered":"2024-08-18T13:51:34.672086+0000","last_clean":"2024-08-18T13:51:34.672086+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:51:34.672086+0000","last_undegraded":"2024-08-18T13:51:34.672086+0000","last_fullsized":"2024-08-18T13:51:34.672086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:56.649240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":128,"num_write_kb":12417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"54'91","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.768186+0000","last_change":"2024-08-18T13:43:56.154716+0000","last_active":"2024-08-18T13:51:51.768186+0000","last_peered":"2024-08-18T13:51:51.768186+0000","last_clean":"2024-08-18T13:51:51.768186+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:51:51.768186+0000","last_undegraded":"2024-08-18T13:51:51.768186+0000","last_fullsized":"2024-08-18T13:51:51.768186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053680339999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"54'664","reported_seq":3634,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.568805+0000","last_change":"2024-08-18T13:43:55.917879+0000","last_active":"2024-08-18T13:51:49.568805+0000","last_peered":"2024-08-18T13:51:49.568805+0000","last_clean":"2024-08-18T13:51:49.568805+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:51:49.568805+0000","last_undegraded":"2024-08-18T13:51:49.568805+0000","last_fullsized":"2024-08-18T13:51:49.568805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:06:31.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035122529999999999,"stat_sum":{"num_bytes":30328916,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":908,"num_read_kb":29115,"num_write":663,"num_write_kb":34071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"54'14","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:38.993056+0000","last_change":"2024-08-18T13:39:34.397638+0000","last_active":"2024-08-18T13:51:38.993056+0000","last_peered":"2024-08-18T13:51:38.993056+0000","last_clean":"2024-08-18T13:51:38.993056+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:51:38.993056+0000","last_undegraded":"2024-08-18T13:51:38.993056+0000","last_fullsized":"2024-08-18T13:51:38.993056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:41:01.966200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":19,"num_write_kb":4249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'85","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757614+0000","last_change":"2024-08-18T13:43:55.555770+0000","last_active":"2024-08-18T13:51:51.757614+0000","last_peered":"2024-08-18T13:51:51.757614+0000","last_clean":"2024-08-18T13:51:51.757614+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:51:51.757614+0000","last_undegraded":"2024-08-18T13:51:51.757614+0000","last_fullsized":"2024-08-18T13:51:51.757614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072935329999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"54'656","reported_seq":3234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.680798+0000","last_change":"2024-08-18T13:43:55.555171+0000","last_active":"2024-08-18T13:51:49.680798+0000","last_peered":"2024-08-18T13:51:49.680798+0000","last_clean":"2024-08-18T13:51:49.680798+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:51:49.680798+0000","last_undegraded":"2024-08-18T13:51:49.680798+0000","last_fullsized":"2024-08-18T13:51:49.680798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:50:29.103230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063112100000000003,"stat_sum":{"num_bytes":21201155,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":516,"num_read_kb":24278,"num_write":657,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"54'107","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764519+0000","last_change":"2024-08-18T13:43:55.959563+0000","last_active":"2024-08-18T13:51:51.764519+0000","last_peered":"2024-08-18T13:51:51.764519+0000","last_clean":"2024-08-18T13:51:51.764519+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:51:51.764519+0000","last_undegraded":"2024-08-18T13:51:51.764519+0000","last_fullsized":"2024-08-18T13:51:51.764519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:49:37.936594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045422170000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"54'110","reported_seq":2173,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776910+0000","last_change":"2024-08-18T13:43:55.919976+0000","last_active":"2024-08-18T13:51:51.776910+0000","last_peered":"2024-08-18T13:51:51.776910+0000","last_clean":"2024-08-18T13:51:51.776910+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:51:51.776910+0000","last_undegraded":"2024-08-18T13:51:51.776910+0000","last_fullsized":"2024-08-18T13:51:51.776910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:29.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032950929999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"54'101","reported_seq":2174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.757889+0000","last_change":"2024-08-18T13:43:55.863304+0000","last_active":"2024-08-18T13:51:51.757889+0000","last_peered":"2024-08-18T13:51:51.757889+0000","last_clean":"2024-08-18T13:51:51.757889+0000","last_became_active":"2024-08-18T13:40:16.610568+0000","last_became_peered":"2024-08-18T13:40:16.610568+0000","last_unstale":"2024-08-18T13:51:51.757889+0000","last_undegraded":"2024-08-18T13:51:51.757889+0000","last_fullsized":"2024-08-18T13:51:51.757889+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:53:53.687401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066503589999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"54'99","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766455+0000","last_change":"2024-08-18T13:43:56.153594+0000","last_active":"2024-08-18T13:51:51.766455+0000","last_peered":"2024-08-18T13:51:51.766455+0000","last_clean":"2024-08-18T13:51:51.766455+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:51:51.766455+0000","last_undegraded":"2024-08-18T13:51:51.766455+0000","last_fullsized":"2024-08-18T13:51:51.766455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056450420000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"54'80","reported_seq":2143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764106+0000","last_change":"2024-08-18T13:43:55.554865+0000","last_active":"2024-08-18T13:51:51.764106+0000","last_peered":"2024-08-18T13:51:51.764106+0000","last_clean":"2024-08-18T13:51:51.764106+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:51:51.764106+0000","last_undegraded":"2024-08-18T13:51:51.764106+0000","last_fullsized":"2024-08-18T13:51:51.764106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:14:11.457615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068042459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"54'74","reported_seq":2139,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761493+0000","last_change":"2024-08-18T13:43:56.142479+0000","last_active":"2024-08-18T13:51:51.761493+0000","last_peered":"2024-08-18T13:51:51.761493+0000","last_clean":"2024-08-18T13:51:51.761493+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:51:51.761493+0000","last_undegraded":"2024-08-18T13:51:51.761493+0000","last_fullsized":"2024-08-18T13:51:51.761493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079157710000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"54'109","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779808+0000","last_change":"2024-08-18T13:43:55.959839+0000","last_active":"2024-08-18T13:51:51.779808+0000","last_peered":"2024-08-18T13:51:51.779808+0000","last_clean":"2024-08-18T13:51:51.779808+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:51:51.779808+0000","last_undegraded":"2024-08-18T13:51:51.779808+0000","last_fullsized":"2024-08-18T13:51:51.779808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:10:29.428131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029546540000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"54'85","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755227+0000","last_change":"2024-08-18T13:43:55.555415+0000","last_active":"2024-08-18T13:51:51.755227+0000","last_peered":"2024-08-18T13:51:51.755227+0000","last_clean":"2024-08-18T13:51:51.755227+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:51:51.755227+0000","last_undegraded":"2024-08-18T13:51:51.755227+0000","last_fullsized":"2024-08-18T13:51:51.755227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:45.775194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055234810000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"54'92","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762023+0000","last_change":"2024-08-18T13:43:55.916616+0000","last_active":"2024-08-18T13:51:51.762023+0000","last_peered":"2024-08-18T13:51:51.762023+0000","last_clean":"2024-08-18T13:51:51.762023+0000","last_became_active":"2024-08-18T13:40:16.614889+0000","last_became_peered":"2024-08-18T13:40:16.614889+0000","last_unstale":"2024-08-18T13:51:51.762023+0000","last_undegraded":"2024-08-18T13:51:51.762023+0000","last_fullsized":"2024-08-18T13:51:51.762023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:37:46.326075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033397829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"54'92","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761272+0000","last_change":"2024-08-18T13:43:55.554199+0000","last_active":"2024-08-18T13:51:51.761272+0000","last_peered":"2024-08-18T13:51:51.761272+0000","last_clean":"2024-08-18T13:51:51.761272+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:51:51.761272+0000","last_undegraded":"2024-08-18T13:51:51.761272+0000","last_fullsized":"2024-08-18T13:51:51.761272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:24:27.271524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070000389999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"54'95","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.780212+0000","last_change":"2024-08-18T13:43:55.863255+0000","last_active":"2024-08-18T13:51:51.780212+0000","last_peered":"2024-08-18T13:51:51.780212+0000","last_clean":"2024-08-18T13:51:51.780212+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:51:51.780212+0000","last_undegraded":"2024-08-18T13:51:51.780212+0000","last_fullsized":"2024-08-18T13:51:51.780212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:53:31.568185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070451519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"54'88","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781337+0000","last_change":"2024-08-18T13:43:56.152879+0000","last_active":"2024-08-18T13:51:51.781337+0000","last_peered":"2024-08-18T13:51:51.781337+0000","last_clean":"2024-08-18T13:51:51.781337+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:51:51.781337+0000","last_undegraded":"2024-08-18T13:51:51.781337+0000","last_fullsized":"2024-08-18T13:51:51.781337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070254549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.759908+0000","last_change":"2024-08-18T13:43:55.956991+0000","last_active":"2024-08-18T13:51:51.759908+0000","last_peered":"2024-08-18T13:51:51.759908+0000","last_clean":"2024-08-18T13:51:51.759908+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:51:51.759908+0000","last_undegraded":"2024-08-18T13:51:51.759908+0000","last_fullsized":"2024-08-18T13:51:51.759908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:36.301192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003261548,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"54'85","reported_seq":2144,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758780+0000","last_change":"2024-08-18T13:43:56.141939+0000","last_active":"2024-08-18T13:51:51.758780+0000","last_peered":"2024-08-18T13:51:51.758780+0000","last_clean":"2024-08-18T13:51:51.758780+0000","last_became_active":"2024-08-18T13:40:16.612760+0000","last_became_peered":"2024-08-18T13:40:16.612760+0000","last_unstale":"2024-08-18T13:51:51.758780+0000","last_undegraded":"2024-08-18T13:51:51.758780+0000","last_fullsized":"2024-08-18T13:51:51.758780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076223990000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"54'88","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767420+0000","last_change":"2024-08-18T13:43:55.917355+0000","last_active":"2024-08-18T13:51:51.767420+0000","last_peered":"2024-08-18T13:51:51.767420+0000","last_clean":"2024-08-18T13:51:51.767420+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:51:51.767420+0000","last_undegraded":"2024-08-18T13:51:51.767420+0000","last_fullsized":"2024-08-18T13:51:51.767420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:20:52.083973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027557649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"54'94","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779547+0000","last_change":"2024-08-18T13:43:55.860909+0000","last_active":"2024-08-18T13:51:51.779547+0000","last_peered":"2024-08-18T13:51:51.779547+0000","last_clean":"2024-08-18T13:51:51.779547+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:51:51.779547+0000","last_undegraded":"2024-08-18T13:51:51.779547+0000","last_fullsized":"2024-08-18T13:51:51.779547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:43:31.450650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069473399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"54'96","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775325+0000","last_change":"2024-08-18T13:43:55.860127+0000","last_active":"2024-08-18T13:51:51.775325+0000","last_peered":"2024-08-18T13:51:51.775325+0000","last_clean":"2024-08-18T13:51:51.775325+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:51:51.775325+0000","last_undegraded":"2024-08-18T13:51:51.775325+0000","last_fullsized":"2024-08-18T13:51:51.775325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:20:25.176226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070196240000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"54'85","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761280+0000","last_change":"2024-08-18T13:43:56.136762+0000","last_active":"2024-08-18T13:51:51.761280+0000","last_peered":"2024-08-18T13:51:51.761280+0000","last_clean":"2024-08-18T13:51:51.761280+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:51:51.761280+0000","last_undegraded":"2024-08-18T13:51:51.761280+0000","last_fullsized":"2024-08-18T13:51:51.761280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004509931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"54'92","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762313+0000","last_change":"2024-08-18T13:43:56.154301+0000","last_active":"2024-08-18T13:51:51.762313+0000","last_peered":"2024-08-18T13:51:51.762313+0000","last_clean":"2024-08-18T13:51:51.762313+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:51:51.762313+0000","last_undegraded":"2024-08-18T13:51:51.762313+0000","last_fullsized":"2024-08-18T13:51:51.762313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062343609999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"54'81","reported_seq":2144,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767251+0000","last_change":"2024-08-18T13:43:55.554507+0000","last_active":"2024-08-18T13:51:51.767251+0000","last_peered":"2024-08-18T13:51:51.767251+0000","last_clean":"2024-08-18T13:51:51.767251+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:51:51.767251+0000","last_undegraded":"2024-08-18T13:51:51.767251+0000","last_fullsized":"2024-08-18T13:51:51.767251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:01:22.609168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072121939999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"54'77","reported_seq":2142,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767471+0000","last_change":"2024-08-18T13:43:55.558437+0000","last_active":"2024-08-18T13:51:51.767471+0000","last_peered":"2024-08-18T13:51:51.767471+0000","last_clean":"2024-08-18T13:51:51.767471+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:51:51.767471+0000","last_undegraded":"2024-08-18T13:51:51.767471+0000","last_fullsized":"2024-08-18T13:51:51.767471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057680220000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"54'90","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776234+0000","last_change":"2024-08-18T13:43:55.958601+0000","last_active":"2024-08-18T13:51:51.776234+0000","last_peered":"2024-08-18T13:51:51.776234+0000","last_clean":"2024-08-18T13:51:51.776234+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:51:51.776234+0000","last_undegraded":"2024-08-18T13:51:51.776234+0000","last_fullsized":"2024-08-18T13:51:51.776234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:12:34.835572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049025989999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"54'98","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779265+0000","last_change":"2024-08-18T13:43:55.959710+0000","last_active":"2024-08-18T13:51:51.779265+0000","last_peered":"2024-08-18T13:51:51.779265+0000","last_clean":"2024-08-18T13:51:51.779265+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:51:51.779265+0000","last_undegraded":"2024-08-18T13:51:51.779265+0000","last_fullsized":"2024-08-18T13:51:51.779265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:20:51.331394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029569129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"54'83","reported_seq":2146,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781497+0000","last_change":"2024-08-18T13:43:55.552920+0000","last_active":"2024-08-18T13:51:51.781497+0000","last_peered":"2024-08-18T13:51:51.781497+0000","last_clean":"2024-08-18T13:51:51.781497+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:51:51.781497+0000","last_undegraded":"2024-08-18T13:51:51.781497+0000","last_fullsized":"2024-08-18T13:51:51.781497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:29:05.518224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062586680000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"54'74","reported_seq":2137,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777962+0000","last_change":"2024-08-18T13:43:55.551745+0000","last_active":"2024-08-18T13:51:51.777962+0000","last_peered":"2024-08-18T13:51:51.777962+0000","last_clean":"2024-08-18T13:51:51.777962+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:51:51.777962+0000","last_undegraded":"2024-08-18T13:51:51.777962+0000","last_fullsized":"2024-08-18T13:51:51.777962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:47:30.299021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064056340000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"54'105","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774468+0000","last_change":"2024-08-18T13:43:55.554037+0000","last_active":"2024-08-18T13:51:51.774468+0000","last_peered":"2024-08-18T13:51:51.774468+0000","last_clean":"2024-08-18T13:51:51.774468+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:51:51.774468+0000","last_undegraded":"2024-08-18T13:51:51.774468+0000","last_fullsized":"2024-08-18T13:51:51.774468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:39:21.440229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050911749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"54'93","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774885+0000","last_change":"2024-08-18T13:43:55.862998+0000","last_active":"2024-08-18T13:51:51.774885+0000","last_peered":"2024-08-18T13:51:51.774885+0000","last_clean":"2024-08-18T13:51:51.774885+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:51:51.774885+0000","last_undegraded":"2024-08-18T13:51:51.774885+0000","last_fullsized":"2024-08-18T13:51:51.774885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:10:27.011595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068654359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"54'102","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.776207+0000","last_change":"2024-08-18T13:43:55.550186+0000","last_active":"2024-08-18T13:51:51.776207+0000","last_peered":"2024-08-18T13:51:51.776207+0000","last_clean":"2024-08-18T13:51:51.776207+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:51:51.776207+0000","last_undegraded":"2024-08-18T13:51:51.776207+0000","last_fullsized":"2024-08-18T13:51:51.776207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:16.033798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055002549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"54'76","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.774945+0000","last_change":"2024-08-18T13:43:55.859954+0000","last_active":"2024-08-18T13:51:51.774945+0000","last_peered":"2024-08-18T13:51:51.774945+0000","last_clean":"2024-08-18T13:51:51.774945+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:51:51.774945+0000","last_undegraded":"2024-08-18T13:51:51.774945+0000","last_fullsized":"2024-08-18T13:51:51.774945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:33:08.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069338170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"54'90","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762141+0000","last_change":"2024-08-18T13:43:56.150131+0000","last_active":"2024-08-18T13:51:51.762141+0000","last_peered":"2024-08-18T13:51:51.762141+0000","last_clean":"2024-08-18T13:51:51.762141+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:51:51.762141+0000","last_undegraded":"2024-08-18T13:51:51.762141+0000","last_fullsized":"2024-08-18T13:51:51.762141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044622389999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"54'76","reported_seq":2141,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763906+0000","last_change":"2024-08-18T13:43:56.136824+0000","last_active":"2024-08-18T13:51:51.763906+0000","last_peered":"2024-08-18T13:51:51.763906+0000","last_clean":"2024-08-18T13:51:51.763906+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:51:51.763906+0000","last_undegraded":"2024-08-18T13:51:51.763906+0000","last_fullsized":"2024-08-18T13:51:51.763906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035965799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"54'86","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778822+0000","last_change":"2024-08-18T13:43:55.552484+0000","last_active":"2024-08-18T13:51:51.778822+0000","last_peered":"2024-08-18T13:51:51.778822+0000","last_clean":"2024-08-18T13:51:51.778822+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:51:51.778822+0000","last_undegraded":"2024-08-18T13:51:51.778822+0000","last_fullsized":"2024-08-18T13:51:51.778822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:14:49.263291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067690759999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":938443088,"num_objects":10906,"num_object_clones":0,"num_object_copies":43624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10906,"num_whiteouts":0,"num_read":21895,"num_read_kb":705026,"num_write":21484,"num_write_kb":1061752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2001649664,"data_stored":2001649664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5769,"ondisk_log_size":5769,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":184592884,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":586,"num_read_kb":685208,"num_write":6418,"num_write_kb":186042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":369246208,"data_stored":369185768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6343,"ondisk_log_size":6343,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509596,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599856,"kb_used_data":294184,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93771984,"statfs":{"total":96636764160,"available":96022511616,"internally_reserved":0,"allocated":301244416,"data_stored":300944920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2999999999999998,"5min":1.026,"15min":0.878},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":25.744,"5min":25.744,"15min":25.744},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":0.84999999999999998,"15min":0.82599999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.33100000000000002},"max":{"1min":8.1679999999999993,"5min":8.1679999999999993,"15min":8.1679999999999993},"last":0.79000000000000004}]},{"osd":1,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":0.90700000000000003,"15min":0.98599999999999999},"min":{"1min":0.5,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":10.151999999999999,"5min":10.151999999999999,"15min":39.203000000000003},"last":0.749},{"interface":"front","average":{"1min":2.323,"5min":1.111,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":25.852,"5min":25.852,"15min":25.852},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":1.121,"15min":0.95099999999999996},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.39400000000000002},"max":{"1min":25.818999999999999,"5min":25.818999999999999,"15min":25.818999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.85199999999999998},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.35799999999999998},"max":{"1min":1.133,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.629}]},{"osd":3,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.0840000000000001,"15min":1.135},"min":{"1min":0.64700000000000002,"5min":0.42699999999999999,"15min":0.377},"max":{"1min":17.300999999999998,"5min":17.300999999999998,"15min":42.905000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":2.4689999999999999,"5min":1.238,"15min":1.115},"min":{"1min":0.75600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":25.786999999999999,"5min":25.786999999999999,"15min":25.786999999999999},"last":1.0880000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.29699999999999999},"max":{"1min":8.3870000000000005,"5min":8.3870000000000005,"15min":8.3870000000000005},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.171,"15min":1.23},"min":{"1min":0.79800000000000004,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":17.056999999999999,"5min":17.056999999999999,"15min":52.673000000000002},"last":0.68000000000000005}]},{"osd":5,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":1.274,"15min":1.119},"min":{"1min":0.44,"5min":0.44,"15min":0.33700000000000002},"max":{"1min":25.881,"5min":25.881,"15min":25.881},"last":1.129},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.1080000000000001,"15min":1.073},"min":{"1min":0.73899999999999999,"5min":0.54200000000000004,"15min":0.37},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":0.86399999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.1659999999999999,"15min":1.1259999999999999},"min":{"1min":0.83799999999999997,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":8.0510000000000002,"5min":8.0510000000000002,"15min":8.0510000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.1990000000000001,"15min":1.26},"min":{"1min":0.85499999999999998,"5min":0.49399999999999999,"15min":0.435},"max":{"1min":10.523,"5min":10.523,"15min":39.161000000000001},"last":0.83699999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610072,"kb_used_data":312720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93761768,"statfs":{"total":96636764160,"available":96012050432,"internally_reserved":0,"allocated":320225280,"data_stored":319926507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76500000000000001,"15min":0.88},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.165,"5min":1.21,"15min":4.532},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.248},"max":{"1min":1.258,"5min":1.3200000000000001,"15min":1.9650000000000001},"last":0.98499999999999999}]},{"osd":1,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.92000000000000004},"min":{"1min":0.42399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2749999999999999,"5min":1.294,"15min":4.468},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.91900000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.30499999999999999},"max":{"1min":1.1990000000000001,"5min":1.286,"15min":2.3620000000000001},"last":0.73099999999999998}]},{"osd":2,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77700000000000002,"15min":0.83899999999999997},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.28000000000000003},"max":{"1min":1.085,"5min":1.177,"15min":1.413},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.97299999999999998},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1060000000000001,"5min":1.468,"15min":4.5},"last":1.0880000000000001}]},{"osd":3,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88700000000000001,"15min":0.94199999999999995},"min":{"1min":0.60899999999999999,"5min":0.41999999999999998,"15min":0.35399999999999998},"max":{"1min":1.1919999999999999,"5min":1.276,"15min":4.3250000000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.91600000000000004},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.32900000000000001},"max":{"1min":1.0920000000000001,"5min":1.306,"15min":1.8069999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92700000000000005,"15min":1.0109999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.228,"5min":1.552,"15min":5.2549999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.7230000000000001,"15min":4.6600000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":1.0129999999999999},"min":{"1min":0.54900000000000004,"5min":0.45400000000000001,"15min":0.373},"max":{"1min":1.179,"5min":1.3160000000000001,"15min":4.4009999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.042,"15min":1.1499999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.272,"5min":1.6879999999999999,"15min":6.1260000000000003},"last":1.2849999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0669999999999999,"15min":1.111},"min":{"1min":0.67600000000000005,"5min":0.55500000000000005,"15min":0.40500000000000003},"max":{"1min":1.3009999999999999,"5min":1.4830000000000001,"15min":1.9299999999999999},"last":1.119},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0609999999999999,"15min":1.1080000000000001},"min":{"1min":0.67500000000000004,"5min":0.46300000000000002,"15min":0.40200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":2.1150000000000002},"last":1.252}]}]},{"osd":4,"up_from":15,"seq":64424509595,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570136,"kb_used_data":272784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93801704,"statfs":{"total":96636764160,"available":96052944896,"internally_reserved":0,"allocated":279330816,"data_stored":279026236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.24299999999999999},"max":{"1min":1.1679999999999999,"5min":1.204,"15min":1.22},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":1.085,"5min":1.339,"15min":1.73},"last":1.157}]},{"osd":1,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82499999999999996,"15min":0.86799999999999999},"min":{"1min":0.54100000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.2909999999999999,"5min":1.415,"15min":3.6299999999999999},"last":0.629},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.36699999999999999,"15min":0.27600000000000002},"max":{"1min":1.0589999999999999,"5min":1.28,"15min":1.5169999999999999},"last":1.194}]},{"osd":2,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.85799999999999998},"min":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.5920000000000001},"last":0.89200000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92100000000000004,"15min":0.91800000000000004},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.309},"max":{"1min":1.111,"5min":1.387,"15min":1.45},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.641,"15min":1.641},"last":1.044}]},{"osd":5,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.313},"max":{"1min":1.3640000000000001,"5min":1.575,"15min":3.7789999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.998,"15min":1.0149999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3129999999999999,"5min":1.754,"15min":1.754},"last":0.94999999999999996}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0580000000000001,"15min":1.079},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.504},"max":{"1min":1.6259999999999999,"5min":1.901,"15min":7.8490000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.016,"15min":1.0629999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2809999999999999,"5min":1.5229999999999999,"15min":6.3380000000000001},"last":1.0780000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40000000000000002},"max":{"1min":1.2470000000000001,"5min":1.7470000000000001,"15min":1.8400000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.151,"15min":1.1819999999999999},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.42399999999999999},"max":{"1min":1.6850000000000001,"5min":2.4569999999999999,"15min":3.7410000000000001},"last":1.0169999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542300,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683436,"kb_used_data":373604,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93688404,"statfs":{"total":96636764160,"available":95936925696,"internally_reserved":0,"allocated":382570496,"data_stored":382269906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.29499999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.89500000000000002,"5min":1.2050000000000001,"15min":1.2170000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67800000000000005,"15min":0.69299999999999995},"min":{"1min":0.44900000000000001,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":1.413,"15min":1.413},"last":0.74299999999999999}]},{"osd":1,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.68600000000000005},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86499999999999999,"5min":1.46,"15min":3.4820000000000002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.751},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.8979999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.48599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1910000000000001,"5min":1.6419999999999999,"15min":2.415},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.87,"15min":0.86899999999999999},"min":{"1min":0.57799999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.2589999999999999,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.92200000000000004}]},{"osd":4,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.173,"5min":1.681,"15min":3.2490000000000001},"last":1.23},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.052},"min":{"1min":0.66300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5189999999999999,"5min":2.4870000000000001,"15min":3.8799999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.996,"15min":4.1059999999999999},"last":1.03},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0089999999999999,"15min":1.05},"min":{"1min":0.63200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1719999999999999,"5min":1.627,"15min":3.2799999999999998},"last":1.0529999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.504,"5min":0.504,"15min":0.41999999999999998},"max":{"1min":1.3480000000000001,"5min":1.833,"15min":2.1080000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.054,"15min":1.0880000000000001},"min":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.573,"15min":2.7469999999999999},"last":1.155}]},{"osd":7,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4419999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.111},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.087,"15min":1.0960000000000001},"min":{"1min":0.71499999999999997,"5min":0.63200000000000001,"15min":0.51100000000000001},"max":{"1min":1.583,"5min":1.9039999999999999,"15min":2.1429999999999998},"last":1.0880000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542299,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530780,"kb_used_data":229268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93841060,"statfs":{"total":96636764160,"available":96093245440,"internally_reserved":0,"allocated":234770432,"data_stored":234477374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.186},"max":{"1min":1.0369999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":1.0960000000000001,"5min":3.27,"15min":3.27},"last":0.55800000000000005}]},{"osd":1,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.038,"5min":1.1040000000000001,"15min":1.696},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":3.2290000000000001,"15min":3.2290000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.78200000000000003},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.253},"max":{"1min":0.995,"5min":1.077,"15min":12.468999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.84399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35899999999999999,"15min":0.27500000000000002},"max":{"1min":1.077,"5min":1.3540000000000001,"15min":1.744},"last":0.71699999999999997}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2709999999999999,"5min":1.681,"15min":1.8480000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.94799999999999995},"min":{"1min":0.46899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2609999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.78700000000000003}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1759999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39000000000000001},"max":{"1min":1.149,"5min":1.498,"15min":1.627},"last":0.92200000000000004}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96899999999999997,"15min":0.98799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.42199999999999999},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.649},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.0309999999999999,"15min":1.0649999999999999},"min":{"1min":0.67000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":1.5820000000000001,"15min":1.905},"last":0.88500000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0549999999999999,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.60399999999999998,"15min":0.436},"max":{"1min":1.3140000000000001,"5min":1.5089999999999999,"15min":1.9490000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.0149999999999999},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.46400000000000002},"max":{"1min":1.343,"5min":1.4379999999999999,"15min":1.7669999999999999},"last":1.002}]}]},{"osd":1,"up_from":14,"seq":60129542299,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677976,"kb_used_data":376464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93693864,"statfs":{"total":96636764160,"available":95942516736,"internally_reserved":0,"allocated":385499136,"data_stored":385199863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.65200000000000002},"min":{"1min":0.28899999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.94899999999999995,"5min":1.3009999999999999,"15min":3.1160000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.72099999999999997},"min":{"1min":0.30599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.107,"5min":1.327,"15min":3.718},"last":0.65800000000000003}]},{"osd":2,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0549999999999999,"5min":4.968,"15min":4.968},"last":0.749},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33400000000000002,"15min":0.20899999999999999},"max":{"1min":1.327,"5min":1.3560000000000001,"15min":9.3309999999999995},"last":0.78600000000000003}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4810000000000001,"15min":1.927},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79400000000000004,"15min":0.84899999999999998},"min":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":1.46,"5min":1.46,"15min":6.8559999999999999},"last":0.83999999999999997}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.28000000000000003},"max":{"1min":1.488,"5min":5.1020000000000003,"15min":5.6529999999999996},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.036,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.63400000000000001,"5min":0.39700000000000002,"15min":0.28599999999999998},"max":{"1min":1.4410000000000001,"5min":1.554,"15min":1.5740000000000001},"last":0.97499999999999998}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3859999999999999,"5min":1.546,"15min":3.7570000000000001},"last":1.264},{"interface":"front","average":{"1min":1,"5min":1.04,"15min":1.1040000000000001},"min":{"1min":0.47799999999999998,"5min":0.46400000000000002,"15min":0.45700000000000002},"max":{"1min":1.591,"5min":5.1799999999999997,"15min":19.027999999999999},"last":1.208}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.40699999999999997},"max":{"1min":1.5529999999999999,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":1.302},{"interface":"front","average":{"1min":1.113,"5min":1.0880000000000001,"15min":1.131},"min":{"1min":0.54900000000000004,"5min":0.49099999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.861},"last":1.0760000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.069,"15min":1.119},"min":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58799999999999997},"max":{"1min":1.383,"5min":1.526,"15min":3.8100000000000001},"last":1.147},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40000000000000002},"max":{"1min":1.431,"5min":5.3099999999999996,"15min":5.3099999999999996},"last":1.3660000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542299,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549780,"kb_used_data":248268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93822060,"statfs":{"total":96636764160,"available":96073789440,"internally_reserved":0,"allocated":254226432,"data_stored":253919977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.65400000000000003},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":5.3159999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.33500000000000002,"5min":0.254,"15min":0.19500000000000001},"max":{"1min":0.93600000000000005,"5min":1.2370000000000001,"15min":3.4809999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.47099999999999997,"5min":0.32600000000000001,"15min":0.26300000000000001},"max":{"1min":0.99199999999999999,"5min":1.222,"15min":3.3029999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.28},"last":0.55300000000000005}]},{"osd":3,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.288,"5min":1.288,"15min":5.399},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86799999999999999,"15min":0.93100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.33900000000000002},"max":{"1min":1.1799999999999999,"5min":1.365,"15min":5.0359999999999996},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.37},"max":{"1min":1.3899999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":8.7799999999999994},"last":1.0640000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96099999999999997,"15min":0.97499999999999998},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.087,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.40300000000000002},"max":{"1min":1.51,"5min":1.52,"15min":1.8879999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.77800000000000002,"5min":0.46500000000000002,"15min":0.374},"max":{"1min":1.423,"5min":1.599,"15min":3.7200000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.042,"15min":1.085},"min":{"1min":0.68600000000000005,"5min":0.58099999999999996,"15min":0.52600000000000002},"max":{"1min":1.55,"5min":1.659,"15min":3.2789999999999999},"last":0.95499999999999996}]},{"osd":7,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.143,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.60599999999999998,"15min":0.40300000000000002},"max":{"1min":1.514,"5min":1.704,"15min":36.295000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.171,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.55300000000000005,"15min":0.42999999999999999},"max":{"1min":1.5740000000000001,"5min":1.706,"15min":3.448},"last":0.99099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575002,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515540,"kb_used_data":214028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93856300,"statfs":{"total":96636764160,"available":96108851200,"internally_reserved":0,"allocated":219164672,"data_stored":218869425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.81100000000000005},"min":{"1min":0.48199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.135,"5min":1.2789999999999999,"15min":1.901},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.81000000000000005,"15min":0.85599999999999998},"min":{"1min":0.63900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":4.0810000000000004},"last":1.2410000000000001}]},{"osd":1,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.82099999999999995},"min":{"1min":0.51900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.101,"5min":1.429,"15min":2.4809999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1799999999999999,"5min":1.456,"15min":1.7749999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.63200000000000001,"5min":0.46999999999999997,"15min":0.41799999999999998},"max":{"1min":1.1419999999999999,"5min":1.524,"15min":1.774},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.874,"15min":0.93899999999999995},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.32500000000000001},"max":{"1min":1.1299999999999999,"5min":1.3160000000000001,"15min":4.1559999999999997},"last":0.89600000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.94399999999999995,"15min":0.97999999999999998},"min":{"1min":0.59799999999999998,"5min":0.54600000000000004,"15min":0.36199999999999999},"max":{"1min":1.3300000000000001,"5min":1.4410000000000001,"15min":4.5510000000000002},"last":1.03},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.92700000000000005,"15min":0.98499999999999999},"min":{"1min":0.72199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.423,"5min":1.4470000000000001,"15min":3.976},"last":0.877}]},{"osd":4,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.7909999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0660000000000001,"15min":1.1120000000000001},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.6839999999999999,"15min":2.532},"last":0.999}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.03,"15min":1.085},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3109999999999999,"5min":1.532,"15min":4.2050000000000001},"last":1.395},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0569999999999999,"15min":1.113},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":2.911},"last":0.82199999999999995}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.089,"15min":1.165},"min":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2989999999999999,"5min":1.659,"15min":2.569},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.1359999999999999,"15min":1.361},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40400000000000003},"max":{"1min":1.407,"5min":1.605,"15min":58.290999999999997},"last":1.4570000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41955328,"data_stored":41943666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50331808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21549056,"data_stored":21547239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34140160,"data_stored":34130811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":54005760,"data_stored":53992421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41947136,"data_stored":41943130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50332308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":74969088,"data_stored":74964385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211058688,"data_stored":211058688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334540800,"data_stored":334540800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212602880,"data_stored":212602880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347811840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":224706560,"data_stored":224706560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176599040,"data_stored":176599040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":269266944,"data_stored":269266944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":225062912,"data_stored":225062912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:52:04.022 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.022 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.023 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.023 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.023 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.023 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.023 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.023 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.024 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.024 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.024 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.024 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.024 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.024 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.024 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.025 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.025 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.025 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.025 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.025 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.025 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.026 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.026 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.026 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.026 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.026 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.026 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.027 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.027 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.027 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.027 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.027 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.027 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.028 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.028 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.028 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.028 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.028 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.028 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.028 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.029 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.029 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.029 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.029 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.029 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.029 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.030 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-18T13:39:21.399940+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=21, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.030 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.030 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.030 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.030 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.030 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.031 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.031 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.031 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.031 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.031 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.031 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.032 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.032 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.032 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.032 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.032 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.032 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.033 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.033 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.033 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.033 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.033 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.033 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.034 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.034 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.034 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.034 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.035 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.035 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.035 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.035 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.035 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.035 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.036 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.036 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.036 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.036 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.036 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.036 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.037 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.037 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.037 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.037 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.037 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.037 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.038 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.038 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.038 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.038 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.038 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.038 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.039 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.039 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.039 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.039 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.039 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.039 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.040 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.040 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.040 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.040 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.040 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.040 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.041 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.041 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.041 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.041 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.041 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.041 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.042 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.042 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.042 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.042 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.042 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.042 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.043 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.043 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.043 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.043 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.043 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.043 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:04.044 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-18T13:52:14.451 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:52:14.454 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:52:24.045 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:52:24.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c347dc700 1 Processor -- start 2024-08-18T13:52:24.155 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c347dc700 1 -- start start 2024-08-18T13:52:24.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c347dc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f4c2c14f800 0x7f4c2c159350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:24.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c347dc700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4c2c0c0d40 0x7f4c2c087fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:24.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c347dc700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c159890 con 0x7f4c2c14f430 2024-08-18T13:52:24.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c347dc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c1557f0 con 0x7f4c2c14f800 2024-08-18T13:52:24.157 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c347dc700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c067e00 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c32578700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f4c2c14f430 0x7f4c2c155c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:46726/0 (socket says 172.21.15.118:46726) 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c32578700 1 -- 172.21.15.118:0/61542987 learned_addr learned my addr 172.21.15.118:0/61542987 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31d77700 1 --2- 172.21.15.118:0/61542987 >> v2:172.21.15.118:3300/0 conn(0x7f4c2c14f800 0x7f4c2c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c32d79700 1 --2- 172.21.15.118:0/61542987 >> v2:172.21.15.178:3300/0 conn(0x7f4c2c0c0d40 0x7f4c2c087fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31576700 1 -- 172.21.15.118:0/61542987 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745565894 0 0) 0x7f4c24001490 con 0x7f4c2c14f430 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31576700 1 -- 172.21.15.118:0/61542987 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c18003570 con 0x7f4c2c14f430 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31d77700 1 -- 172.21.15.118:0/61542987 >> v2:172.21.15.178:3300/0 conn(0x7f4c2c0c0d40 msgr2=0x7f4c2c087fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:52:24.158 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31d77700 1 --2- 172.21.15.118:0/61542987 >> v2:172.21.15.178:3300/0 conn(0x7f4c2c0c0d40 0x7f4c2c087fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.159 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31576700 1 -- 172.21.15.118:0/61542987 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 545338501 0 0) 0x7f4c18003570 con 0x7f4c2c14f430 2024-08-18T13:52:24.159 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31d77700 1 -- 172.21.15.118:0/61542987 >> v1:172.21.15.118:6789/0 conn(0x7f4c2c14f430 legacy=0x7f4c2c155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:24.159 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.149+0000 7f4c31d77700 1 -- 172.21.15.118:0/61542987 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0680a0 con 0x7f4c2c14f800 2024-08-18T13:52:24.160 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c31d77700 1 --2- 172.21.15.118:0/61542987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c14f800 0x7f4c2c159350 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0c87f0 tx=0x7f4c1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0407ffbd45c439 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:24.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c31576700 1 -- 172.21.15.118:0/61542987 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24001980 con 0x7f4c2c14f800 2024-08-18T13:52:24.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c31576700 1 -- 172.21.15.118:0/61542987 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c011420 con 0x7f4c2c14f800 2024-08-18T13:52:24.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c31576700 1 -- 172.21.15.118:0/61542987 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0117c0 con 0x7f4c2c14f800 2024-08-18T13:52:24.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 -- 172.21.15.118:0/61542987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c14f800 msgr2=0x7f4c2c159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:24.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 --2- 172.21.15.118:0/61542987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c14f800 0x7f4c2c159350 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0c87f0 tx=0x7f4c1c0091a0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 -- 172.21.15.118:0/61542987 shutdown_connections 2024-08-18T13:52:24.161 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 --2- 172.21.15.118:0/61542987 >> v2:172.21.15.178:3300/0 conn(0x7f4c2c0c0d40 0x7f4c2c087fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 --2- 172.21.15.118:0/61542987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c14f800 0x7f4c2c159350 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 -- 172.21.15.118:0/61542987 >> 172.21.15.118:0/61542987 conn(0x7f4c2c068970 msgr2=0x7f4c2c068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:24.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 -- 172.21.15.118:0/61542987 shutdown_connections 2024-08-18T13:52:24.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 -- 172.21.15.118:0/61542987 wait complete. 2024-08-18T13:52:24.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 Processor -- start 2024-08-18T13:52:24.162 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.153+0000 7f4c347dc700 1 -- start start 2024-08-18T13:52:24.164 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c0c0d40 0x7f4c2c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:24.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2c14f430 0x7f4c2c0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:24.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c2c14f800 0x7f4c2c1656f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:24.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0c80d0 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0c6430 con 0x7f4c2c14f430 2024-08-18T13:52:24.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c154d40 con 0x7f4c2c14f800 2024-08-18T13:52:24.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c0c0d40 0x7f4c2c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:24.165 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c0c0d40 0x7f4c2c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:37350/0 (socket says 172.21.15.118:37350) 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 -- 172.21.15.118:0/2944582143 learned_addr learned my addr 172.21.15.118:0/2944582143 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32d79700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c2c14f800 0x7f4c2c1656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c31d77700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2c14f430 0x7f4c2c0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 -- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c2c14f800 msgr2=0x7f4c2c1656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c2c14f800 0x7f4c2c1656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 -- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2c14f430 msgr2=0x7f4c2c0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2c14f430 0x7f4c2c0aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.166 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 -- 172.21.15.118:0/2944582143 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c009420 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c31d77700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2c14f430 0x7f4c2c0aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:52:24.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c32578700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c0c0d40 0x7f4c2c0c7b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400cd80 tx=0x7f4c24007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:24.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c227fc700 1 -- 172.21.15.118:0/2944582143 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400ce80 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c227fc700 1 -- 172.21.15.118:0/2944582143 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24004dc0 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c155270 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.167 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c227fc700 1 -- 172.21.15.118:0/2944582143 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400ab30 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.168 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c0c5680 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.168 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.157+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c0642f0 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.161+0000 7f4c227fc700 1 -- 172.21.15.118:0/2944582143 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4c24007150 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.161+0000 7f4c227fc700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4c10060860 0x7f4c10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:24.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.161+0000 7f4c227fc700 1 -- 172.21.15.118:0/2944582143 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f4c2407cb00 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.161+0000 7f4c31d77700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4c10060860 0x7f4c10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:24.169 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.161+0000 7f4c31d77700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4c10060860 0x7f4c10062d10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0c87f0 tx=0x7f4c1c016000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:24.175 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.169+0000 7f4c227fc700 1 -- 172.21.15.118:0/2944582143 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4c2404b480 con 0x7f4c2c0c0d40 2024-08-18T13:52:24.346 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.341+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c2c159890 con 0x7f4c10060860 2024-08-18T13:52:24.365 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.361+0000 7f4c227fc700 1 -- 172.21.15.118:0/2944582143 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371730 (secure 0 0 0) 0x7f4c2c159890 con 0x7f4c10060860 2024-08-18T13:52:24.371 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:52:24.374 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4c10060860 msgr2=0x7f4c10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:24.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4c10060860 0x7f4c10062d10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0c87f0 tx=0x7f4c1c016000 comp rx=0 tx=0).stop 2024-08-18T13:52:24.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c0c0d40 msgr2=0x7f4c2c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:24.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c0c0d40 0x7f4c2c0c7b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400cd80 tx=0x7f4c24007c70 comp rx=0 tx=0).stop 2024-08-18T13:52:24.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 shutdown_connections 2024-08-18T13:52:24.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f4c10060860 0x7f4c10062d10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c2c14f800 0x7f4c2c1656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.375 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2c14f430 0x7f4c2c0aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 --2- 172.21.15.118:0/2944582143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4c2c0c0d40 0x7f4c2c0c7b90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:24.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 >> 172.21.15.118:0/2944582143 conn(0x7f4c2c068970 msgr2=0x7f4c2c158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:24.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 shutdown_connections 2024-08-18T13:52:24.376 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:24.365+0000 7f4c347dc700 1 -- 172.21.15.118:0/2944582143 wait complete. 2024-08-18T13:52:24.376 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:52:24.387 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-08-18T13:52:23.498553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1123881704,"num_objects":11047,"num_object_clones":0,"num_object_copies":43906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11047,"num_whiteouts":0,"num_read":22597,"num_read_kb":1390334,"num_write":28078,"num_write_kb":1250468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":544529,"num_omap_keys":1040,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33889,"ondisk_log_size":33889,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4738080,"kb_used_data":2321824,"kb_used_omap":13,"kb_used_meta":2416178,"kb_avail":750236640,"statfs":{"total":773094113280,"available":768242319360,"internally_reserved":0,"allocated":2377547776,"data_stored":2375144936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2474166976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":156642,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":339003,"num_omap_keys":646,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005252"},"pg_stats":[{"pgid":"2.1f","version":"54'156","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.111340+0000","last_change":"2024-08-18T13:52:07.111260+0000","last_active":"2024-08-18T13:52:07.111340+0000","last_peered":"2024-08-18T13:52:07.111340+0000","last_clean":"2024-08-18T13:52:07.111340+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:52:07.111340+0000","last_undegraded":"2024-08-18T13:52:07.111340+0000","last_fullsized":"2024-08-18T13:52:07.111340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'156","last_scrub_stamp":"2024-08-18T13:52:07.111121+0000","last_deep_scrub":"54'156","last_deep_scrub_stamp":"2024-08-18T13:52:07.111121+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.111121+0000","objects_scrubbed":5,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:38.124367+0000","scrub_duration":0.025498494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389178,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":28696,"num_write":150,"num_write_kb":8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11702,"num_omap_keys":22,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.768077+0000","last_change":"2024-08-18T13:43:55.555634+0000","last_active":"2024-08-18T13:51:51.768077+0000","last_peered":"2024-08-18T13:51:51.768077+0000","last_clean":"2024-08-18T13:51:51.768077+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:51:51.768077+0000","last_undegraded":"2024-08-18T13:51:51.768077+0000","last_fullsized":"2024-08-18T13:51:51.768077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064850389999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"54'676","reported_seq":3752,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.851363+0000","last_change":"2024-08-18T13:43:55.552107+0000","last_active":"2024-08-18T13:51:49.851363+0000","last_peered":"2024-08-18T13:51:49.851363+0000","last_clean":"2024-08-18T13:51:49.851363+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:51:49.851363+0000","last_undegraded":"2024-08-18T13:51:49.851363+0000","last_fullsized":"2024-08-18T13:51:49.851363+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067303959999999996,"stat_sum":{"num_bytes":37566158,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":1013,"num_read_kb":36248,"num_write":673,"num_write_kb":41139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"54'79","reported_seq":138,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.012943+0000","last_change":"2024-08-18T13:52:14.012891+0000","last_active":"2024-08-18T13:52:14.012943+0000","last_peered":"2024-08-18T13:52:14.012943+0000","last_clean":"2024-08-18T13:52:14.012943+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:52:14.012943+0000","last_undegraded":"2024-08-18T13:52:14.012943+0000","last_fullsized":"2024-08-18T13:52:14.012943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'79","last_scrub_stamp":"2024-08-18T13:52:14.012738+0000","last_deep_scrub":"54'79","last_deep_scrub_stamp":"2024-08-18T13:52:14.012738+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.012738+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:27:28.341973+0000","scrub_duration":0.015505003999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":75,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"54'87","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:01.076425+0000","last_change":"2024-08-18T13:52:01.076369+0000","last_active":"2024-08-18T13:52:01.076425+0000","last_peered":"2024-08-18T13:52:01.076425+0000","last_clean":"2024-08-18T13:52:01.076425+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:52:01.076425+0000","last_undegraded":"2024-08-18T13:52:01.076425+0000","last_fullsized":"2024-08-18T13:52:01.076425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_clean_scrub_stamp":"2024-08-18T13:52:01.076152+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:46:50.073762+0000","scrub_duration":0.0039656070000000003,"objects_trimmed":0,"snaptrim_duration":0.0075430439999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"54'705","reported_seq":3584,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.827339+0000","last_change":"2024-08-18T13:52:18.827164+0000","last_active":"2024-08-18T13:52:18.827339+0000","last_peered":"2024-08-18T13:52:18.827339+0000","last_clean":"2024-08-18T13:52:18.827339+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:52:18.827339+0000","last_undegraded":"2024-08-18T13:52:18.827339+0000","last_fullsized":"2024-08-18T13:52:18.827339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'705","last_scrub_stamp":"2024-08-18T13:52:18.826906+0000","last_deep_scrub":"54'705","last_deep_scrub_stamp":"2024-08-18T13:52:18.826906+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.826906+0000","objects_scrubbed":370,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:26:33.481234+0000","scrub_duration":0.157720147,"objects_trimmed":0,"snaptrim_duration":0.0075371290000000001,"stat_sum":{"num_bytes":29851183,"num_objects":370,"num_object_clones":0,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":792,"num_read_kb":20881,"num_write":708,"num_write_kb":37566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"54'194","reported_seq":260,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:48.188423+0000","last_change":"2024-08-18T13:39:34.395101+0000","last_active":"2024-08-18T13:51:48.188423+0000","last_peered":"2024-08-18T13:51:48.188423+0000","last_clean":"2024-08-18T13:51:48.188423+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:51:48.188423+0000","last_undegraded":"2024-08-18T13:51:48.188423+0000","last_fullsized":"2024-08-18T13:51:48.188423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:33.349883+0000","last_clean_scrub_stamp":"2024-08-18T13:39:33.349883+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":69650,"num_write":192,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"54'90","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775288+0000","last_change":"2024-08-18T13:43:55.857350+0000","last_active":"2024-08-18T13:51:51.775288+0000","last_peered":"2024-08-18T13:51:51.775288+0000","last_clean":"2024-08-18T13:51:51.775288+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:51:51.775288+0000","last_undegraded":"2024-08-18T13:51:51.775288+0000","last_fullsized":"2024-08-18T13:51:51.775288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046085359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"54'680","reported_seq":3481,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.517007+0000","last_change":"2024-08-18T13:43:55.557895+0000","last_active":"2024-08-18T13:51:49.517007+0000","last_peered":"2024-08-18T13:51:49.517007+0000","last_clean":"2024-08-18T13:51:49.517007+0000","last_became_active":"2024-08-18T13:40:16.608743+0000","last_became_peered":"2024-08-18T13:40:16.608743+0000","last_unstale":"2024-08-18T13:51:49.517007+0000","last_undegraded":"2024-08-18T13:51:49.517007+0000","last_fullsized":"2024-08-18T13:51:49.517007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080275980000000004,"stat_sum":{"num_bytes":28774140,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":727,"num_read_kb":13856,"num_write":680,"num_write_kb":28426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"54'27","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.933094+0000","last_change":"2024-08-18T13:52:15.933048+0000","last_active":"2024-08-18T13:52:15.933094+0000","last_peered":"2024-08-18T13:52:15.933094+0000","last_clean":"2024-08-18T13:52:15.933094+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:52:15.933094+0000","last_undegraded":"2024-08-18T13:52:15.933094+0000","last_fullsized":"2024-08-18T13:52:15.933094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'27","last_scrub_stamp":"2024-08-18T13:52:15.932932+0000","last_deep_scrub":"54'27","last_deep_scrub_stamp":"2024-08-18T13:52:15.932932+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.932932+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:50:28.024428+0000","scrub_duration":0.016823828999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":25,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12356,"num_omap_keys":25,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"54'87","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777134+0000","last_change":"2024-08-18T13:43:55.862914+0000","last_active":"2024-08-18T13:51:51.777134+0000","last_peered":"2024-08-18T13:51:51.777134+0000","last_clean":"2024-08-18T13:51:51.777134+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:51:51.777134+0000","last_undegraded":"2024-08-18T13:51:51.777134+0000","last_fullsized":"2024-08-18T13:51:51.777134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065299970000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"54'674","reported_seq":2903,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.468005+0000","last_change":"2024-08-18T13:43:55.555499+0000","last_active":"2024-08-18T13:51:49.468005+0000","last_peered":"2024-08-18T13:51:49.468005+0000","last_clean":"2024-08-18T13:51:49.468005+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:51:49.468005+0000","last_undegraded":"2024-08-18T13:51:49.468005+0000","last_fullsized":"2024-08-18T13:51:49.468005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078514629999999995,"stat_sum":{"num_bytes":11851705,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":166,"num_read_kb":1355,"num_write":673,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"54'88","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755593+0000","last_change":"2024-08-18T13:43:55.954106+0000","last_active":"2024-08-18T13:51:51.755593+0000","last_peered":"2024-08-18T13:51:51.755593+0000","last_clean":"2024-08-18T13:51:51.755593+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:51:51.755593+0000","last_undegraded":"2024-08-18T13:51:51.755593+0000","last_fullsized":"2024-08-18T13:51:51.755593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033699279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"54'252","reported_seq":322,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.940516+0000","last_change":"2024-08-18T13:52:21.940455+0000","last_active":"2024-08-18T13:52:21.940516+0000","last_peered":"2024-08-18T13:52:21.940516+0000","last_clean":"2024-08-18T13:52:21.940516+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:52:21.940516+0000","last_undegraded":"2024-08-18T13:52:21.940516+0000","last_fullsized":"2024-08-18T13:52:21.940516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'252","last_scrub_stamp":"2024-08-18T13:52:21.940309+0000","last_deep_scrub":"54'252","last_deep_scrub_stamp":"2024-08-18T13:52:21.940309+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.940309+0000","objects_scrubbed":9,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:44:34.338028+0000","scrub_duration":0.033709736999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12304619,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":23,"num_read_kb":45068,"num_write":239,"num_write_kb":12181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43761,"num_omap_keys":83,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"54'638","reported_seq":3282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.659037+0000","last_change":"2024-08-18T13:43:55.558893+0000","last_active":"2024-08-18T13:51:49.659037+0000","last_peered":"2024-08-18T13:51:49.659037+0000","last_clean":"2024-08-18T13:51:49.659037+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:51:49.659037+0000","last_undegraded":"2024-08-18T13:51:49.659037+0000","last_fullsized":"2024-08-18T13:51:49.659037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059019540000000001,"stat_sum":{"num_bytes":16782769,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":581,"num_read_kb":14450,"num_write":639,"num_write_kb":24822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"54'87","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.890088+0000","last_change":"2024-08-18T13:52:08.890024+0000","last_active":"2024-08-18T13:52:08.890088+0000","last_peered":"2024-08-18T13:52:08.890088+0000","last_clean":"2024-08-18T13:52:08.890088+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:52:08.890088+0000","last_undegraded":"2024-08-18T13:52:08.890088+0000","last_fullsized":"2024-08-18T13:52:08.890088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:08.889811+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:08.889811+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.889811+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:42:55.362427+0000","scrub_duration":0.0034394989999999999,"objects_trimmed":0,"snaptrim_duration":0.0074252320000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"54'32","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:11.142035+0000","last_change":"2024-08-18T13:52:11.141993+0000","last_active":"2024-08-18T13:52:11.142035+0000","last_peered":"2024-08-18T13:52:11.142035+0000","last_clean":"2024-08-18T13:52:11.142035+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:52:11.142035+0000","last_undegraded":"2024-08-18T13:52:11.142035+0000","last_fullsized":"2024-08-18T13:52:11.142035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'32","last_scrub_stamp":"2024-08-18T13:52:11.141864+0000","last_deep_scrub":"54'32","last_deep_scrub_stamp":"2024-08-18T13:52:11.141864+0000","last_clean_scrub_stamp":"2024-08-18T13:52:11.141864+0000","objects_scrubbed":4,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:40.879760+0000","scrub_duration":0.016960533999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":28,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28048,"num_omap_keys":55,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"54'709","reported_seq":3800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.742550+0000","last_change":"2024-08-18T13:43:55.558557+0000","last_active":"2024-08-18T13:51:49.742550+0000","last_peered":"2024-08-18T13:51:49.742550+0000","last_clean":"2024-08-18T13:51:49.742550+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:51:49.742550+0000","last_undegraded":"2024-08-18T13:51:49.742550+0000","last_fullsized":"2024-08-18T13:51:49.742550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057029610000000003,"stat_sum":{"num_bytes":33413248,"num_objects":347,"num_object_clones":0,"num_object_copies":1388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1028,"num_read_kb":27410,"num_write":707,"num_write_kb":41185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.764813+0000","last_change":"2024-08-18T13:43:55.553624+0000","last_active":"2024-08-18T13:51:51.764813+0000","last_peered":"2024-08-18T13:51:51.764813+0000","last_clean":"2024-08-18T13:51:51.764813+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:51:51.764813+0000","last_undegraded":"2024-08-18T13:51:51.764813+0000","last_fullsized":"2024-08-18T13:51:51.764813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069243719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"54'14","reported_seq":69,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:06.184482+0000","last_change":"2024-08-18T13:52:06.184363+0000","last_active":"2024-08-18T13:52:06.184482+0000","last_peered":"2024-08-18T13:52:06.184482+0000","last_clean":"2024-08-18T13:52:06.184482+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:52:06.184482+0000","last_undegraded":"2024-08-18T13:52:06.184482+0000","last_fullsized":"2024-08-18T13:52:06.184482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'14","last_scrub_stamp":"2024-08-18T13:52:06.184119+0000","last_deep_scrub":"54'14","last_deep_scrub_stamp":"2024-08-18T13:52:06.184119+0000","last_clean_scrub_stamp":"2024-08-18T13:52:06.184119+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:52.279744+0000","scrub_duration":0.013536823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":11,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'683","reported_seq":3571,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.731009+0000","last_change":"2024-08-18T13:43:56.142589+0000","last_active":"2024-08-18T13:51:49.731009+0000","last_peered":"2024-08-18T13:51:49.731009+0000","last_clean":"2024-08-18T13:51:49.731009+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:51:49.731009+0000","last_undegraded":"2024-08-18T13:51:49.731009+0000","last_fullsized":"2024-08-18T13:51:49.731009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079287170000000001,"stat_sum":{"num_bytes":20263748,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":825,"num_read_kb":25932,"num_write":679,"num_write_kb":28331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"54'92","reported_seq":2167,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.649198+0000","last_change":"2024-08-18T13:52:10.649127+0000","last_active":"2024-08-18T13:52:10.649198+0000","last_peered":"2024-08-18T13:52:10.649198+0000","last_clean":"2024-08-18T13:52:10.649198+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:52:10.649198+0000","last_undegraded":"2024-08-18T13:52:10.649198+0000","last_fullsized":"2024-08-18T13:52:10.649198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:10.648930+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:10.648930+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.648930+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:46:12.826484+0000","scrub_duration":0.0036575269999999998,"objects_trimmed":0,"snaptrim_duration":0.0068791520000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"54'110","reported_seq":168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.973413+0000","last_change":"2024-08-18T13:52:18.973337+0000","last_active":"2024-08-18T13:52:18.973413+0000","last_peered":"2024-08-18T13:52:18.973413+0000","last_clean":"2024-08-18T13:52:18.973413+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:52:18.973413+0000","last_undegraded":"2024-08-18T13:52:18.973413+0000","last_fullsized":"2024-08-18T13:52:18.973413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'110","last_scrub_stamp":"2024-08-18T13:52:18.973134+0000","last_deep_scrub":"54'110","last_deep_scrub_stamp":"2024-08-18T13:52:18.973134+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.973134+0000","objects_scrubbed":4,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:40.390138+0000","scrub_duration":0.045608213000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":108,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12312,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"54'617","reported_seq":3453,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.737566+0000","last_change":"2024-08-18T13:43:55.559559+0000","last_active":"2024-08-18T13:51:49.737566+0000","last_peered":"2024-08-18T13:51:49.737566+0000","last_clean":"2024-08-18T13:51:49.737566+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:51:49.737566+0000","last_undegraded":"2024-08-18T13:51:49.737566+0000","last_fullsized":"2024-08-18T13:51:49.737566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004078028,"stat_sum":{"num_bytes":24272349,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":773,"num_read_kb":25488,"num_write":617,"num_write_kb":28093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'93","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.019178+0000","last_change":"2024-08-18T13:52:09.019114+0000","last_active":"2024-08-18T13:52:09.019178+0000","last_peered":"2024-08-18T13:52:09.019178+0000","last_clean":"2024-08-18T13:52:09.019178+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:52:09.019178+0000","last_undegraded":"2024-08-18T13:52:09.019178+0000","last_fullsized":"2024-08-18T13:52:09.019178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'93","last_scrub_stamp":"2024-08-18T13:52:09.018858+0000","last_deep_scrub":"54'93","last_deep_scrub_stamp":"2024-08-18T13:52:09.018858+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.018858+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:55.114483+0000","scrub_duration":0.0031236559999999998,"objects_trimmed":0,"snaptrim_duration":0.0060661889999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"54'114","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.265449+0000","last_change":"2024-08-18T13:52:05.265412+0000","last_active":"2024-08-18T13:52:05.265449+0000","last_peered":"2024-08-18T13:52:05.265449+0000","last_clean":"2024-08-18T13:52:05.265449+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:52:05.265449+0000","last_undegraded":"2024-08-18T13:52:05.265449+0000","last_fullsized":"2024-08-18T13:52:05.265449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'114","last_scrub_stamp":"2024-08-18T13:52:05.265313+0000","last_deep_scrub":"54'114","last_deep_scrub_stamp":"2024-08-18T13:52:05.265313+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.265313+0000","objects_scrubbed":4,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:57:48.580753+0000","scrub_duration":0.015757296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":111,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16148,"num_omap_keys":31,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"54'720","reported_seq":2994,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.759072+0000","last_change":"2024-08-18T13:52:21.758999+0000","last_active":"2024-08-18T13:52:21.759072+0000","last_peered":"2024-08-18T13:52:21.759072+0000","last_clean":"2024-08-18T13:52:21.759072+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:52:21.759072+0000","last_undegraded":"2024-08-18T13:52:21.759072+0000","last_fullsized":"2024-08-18T13:52:21.759072+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'720","last_scrub_stamp":"2024-08-18T13:52:21.758840+0000","last_deep_scrub":"54'720","last_deep_scrub_stamp":"2024-08-18T13:52:21.758840+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.758840+0000","objects_scrubbed":368,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:36.545679+0000","scrub_duration":0.120970787,"objects_trimmed":0,"snaptrim_duration":0.0079796720000000002,"stat_sum":{"num_bytes":17871290,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":187,"num_read_kb":7768,"num_write":717,"num_write_kb":21887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'88","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.783709+0000","last_change":"2024-08-18T13:43:55.555491+0000","last_active":"2024-08-18T13:51:51.783709+0000","last_peered":"2024-08-18T13:51:51.783709+0000","last_clean":"2024-08-18T13:51:51.783709+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:51:51.783709+0000","last_undegraded":"2024-08-18T13:51:51.783709+0000","last_fullsized":"2024-08-18T13:51:51.783709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054340660000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"54'67","reported_seq":128,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.141636+0000","last_change":"2024-08-18T13:52:07.141608+0000","last_active":"2024-08-18T13:52:07.141636+0000","last_peered":"2024-08-18T13:52:07.141636+0000","last_clean":"2024-08-18T13:52:07.141636+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:52:07.141636+0000","last_undegraded":"2024-08-18T13:52:07.141636+0000","last_fullsized":"2024-08-18T13:52:07.141636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'67","last_scrub_stamp":"2024-08-18T13:52:07.141528+0000","last_deep_scrub":"54'67","last_deep_scrub_stamp":"2024-08-18T13:52:07.141528+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.141528+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:03.991301+0000","scrub_duration":0.018011553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":184,"num_write":164,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23450,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"54'714","reported_seq":2930,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.716447+0000","last_change":"2024-08-18T13:43:55.550748+0000","last_active":"2024-08-18T13:51:49.716447+0000","last_peered":"2024-08-18T13:51:49.716447+0000","last_clean":"2024-08-18T13:51:49.716447+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:51:49.716447+0000","last_undegraded":"2024-08-18T13:51:49.716447+0000","last_fullsized":"2024-08-18T13:51:49.716447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057306029999999999,"stat_sum":{"num_bytes":13242151,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":153,"num_read_kb":5341,"num_write":712,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.019415+0000","last_change":"2024-08-18T13:52:20.019359+0000","last_active":"2024-08-18T13:52:20.019415+0000","last_peered":"2024-08-18T13:52:20.019415+0000","last_clean":"2024-08-18T13:52:20.019415+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:52:20.019415+0000","last_undegraded":"2024-08-18T13:52:20.019415+0000","last_fullsized":"2024-08-18T13:52:20.019415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:20.019175+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:20.019175+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.019175+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:46:53.201543+0000","scrub_duration":0.0031831339999999998,"objects_trimmed":0,"snaptrim_duration":0.0064599770000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"54'292","reported_seq":351,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.937801+0000","last_change":"2024-08-18T13:52:16.937745+0000","last_active":"2024-08-18T13:52:16.937801+0000","last_peered":"2024-08-18T13:52:16.937801+0000","last_clean":"2024-08-18T13:52:16.937801+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:52:16.937801+0000","last_undegraded":"2024-08-18T13:52:16.937801+0000","last_fullsized":"2024-08-18T13:52:16.937801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'292","last_scrub_stamp":"2024-08-18T13:52:16.937598+0000","last_deep_scrub":"54'292","last_deep_scrub_stamp":"2024-08-18T13:52:16.937598+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.937598+0000","objects_scrubbed":3,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:46:56.453503+0000","scrub_duration":0.022315498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":12176,"num_write":284,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8544,"num_omap_keys":16,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"54'704","reported_seq":3426,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.674395+0000","last_change":"2024-08-18T13:43:56.151855+0000","last_active":"2024-08-18T13:51:49.674395+0000","last_peered":"2024-08-18T13:51:49.674395+0000","last_clean":"2024-08-18T13:51:49.674395+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:51:49.674395+0000","last_undegraded":"2024-08-18T13:51:49.674395+0000","last_fullsized":"2024-08-18T13:51:49.674395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060004689999999996,"stat_sum":{"num_bytes":28446730,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":659,"num_read_kb":21792,"num_write":702,"num_write_kb":28155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"54'84","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777080+0000","last_change":"2024-08-18T13:43:55.863531+0000","last_active":"2024-08-18T13:51:51.777080+0000","last_peered":"2024-08-18T13:51:51.777080+0000","last_clean":"2024-08-18T13:51:51.777080+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:51:51.777080+0000","last_undegraded":"2024-08-18T13:51:51.777080+0000","last_fullsized":"2024-08-18T13:51:51.777080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006455484,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"54'660","reported_seq":714,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.030412+0000","last_change":"2024-08-18T13:52:16.030333+0000","last_active":"2024-08-18T13:52:16.030412+0000","last_peered":"2024-08-18T13:52:16.030412+0000","last_clean":"2024-08-18T13:52:16.030412+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:52:16.030412+0000","last_undegraded":"2024-08-18T13:52:16.030412+0000","last_fullsized":"2024-08-18T13:52:16.030412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'660","last_scrub_stamp":"2024-08-18T13:52:16.030100+0000","last_deep_scrub":"54'660","last_deep_scrub_stamp":"2024-08-18T13:52:16.030100+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.030100+0000","objects_scrubbed":3,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:55:53.439330+0000","scrub_duration":0.021515340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24518,"num_omap_keys":46,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"54'691","reported_seq":3344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.584569+0000","last_change":"2024-08-18T13:43:56.141004+0000","last_active":"2024-08-18T13:51:49.584569+0000","last_peered":"2024-08-18T13:51:49.584569+0000","last_clean":"2024-08-18T13:51:49.584569+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:51:49.584569+0000","last_undegraded":"2024-08-18T13:51:49.584569+0000","last_fullsized":"2024-08-18T13:51:49.584569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067583870000000002,"stat_sum":{"num_bytes":38683931,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":590,"num_read_kb":16518,"num_write":694,"num_write_kb":42167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"54'97","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.632549+0000","last_change":"2024-08-18T13:52:08.632437+0000","last_active":"2024-08-18T13:52:08.632549+0000","last_peered":"2024-08-18T13:52:08.632549+0000","last_clean":"2024-08-18T13:52:08.632549+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:52:08.632549+0000","last_undegraded":"2024-08-18T13:52:08.632549+0000","last_fullsized":"2024-08-18T13:52:08.632549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'97","last_scrub_stamp":"2024-08-18T13:52:08.632220+0000","last_deep_scrub":"54'97","last_deep_scrub_stamp":"2024-08-18T13:52:08.632220+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.632220+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:30.186903+0000","scrub_duration":0.0038014519999999999,"objects_trimmed":0,"snaptrim_duration":0.005854756,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"54'387","reported_seq":453,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.408711+0000","last_change":"2024-08-18T13:52:09.408667+0000","last_active":"2024-08-18T13:52:09.408711+0000","last_peered":"2024-08-18T13:52:09.408711+0000","last_clean":"2024-08-18T13:52:09.408711+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:52:09.408711+0000","last_undegraded":"2024-08-18T13:52:09.408711+0000","last_fullsized":"2024-08-18T13:52:09.408711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'387","last_scrub_stamp":"2024-08-18T13:52:09.408576+0000","last_deep_scrub":"54'387","last_deep_scrub_stamp":"2024-08-18T13:52:09.408576+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.408576+0000","objects_scrubbed":4,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:46:58.406186+0000","scrub_duration":0.023922603000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":43191,"num_write":382,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16508,"num_omap_keys":31,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"54'661","reported_seq":4461,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.604795+0000","last_change":"2024-08-18T13:43:56.144223+0000","last_active":"2024-08-18T13:51:49.604795+0000","last_peered":"2024-08-18T13:51:49.604795+0000","last_clean":"2024-08-18T13:51:49.604795+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:51:49.604795+0000","last_undegraded":"2024-08-18T13:51:49.604795+0000","last_fullsized":"2024-08-18T13:51:49.604795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067969390000000001,"stat_sum":{"num_bytes":51061167,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":1737,"num_read_kb":43040,"num_write":661,"num_write_kb":53481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"54'83","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.003797+0000","last_change":"2024-08-18T13:52:18.003732+0000","last_active":"2024-08-18T13:52:18.003797+0000","last_peered":"2024-08-18T13:52:18.003797+0000","last_clean":"2024-08-18T13:52:18.003797+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:52:18.003797+0000","last_undegraded":"2024-08-18T13:52:18.003797+0000","last_fullsized":"2024-08-18T13:52:18.003797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'83","last_scrub_stamp":"2024-08-18T13:52:18.003500+0000","last_deep_scrub":"54'83","last_deep_scrub_stamp":"2024-08-18T13:52:18.003500+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.003500+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:20.942863+0000","scrub_duration":0.0033910189999999999,"objects_trimmed":0,"snaptrim_duration":0.0063915389999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'122","reported_seq":296,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:11.436136+0000","last_change":"2024-08-18T13:52:11.436080+0000","last_active":"2024-08-18T13:52:11.436136+0000","last_peered":"2024-08-18T13:52:11.436136+0000","last_clean":"2024-08-18T13:52:11.436136+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:52:11.436136+0000","last_undegraded":"2024-08-18T13:52:11.436136+0000","last_fullsized":"2024-08-18T13:52:11.436136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'122","last_scrub_stamp":"2024-08-18T13:52:11.435881+0000","last_deep_scrub":"54'122","last_deep_scrub_stamp":"2024-08-18T13:52:11.435881+0000","last_clean_scrub_stamp":"2024-08-18T13:52:11.435881+0000","objects_scrubbed":2,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:53:26.277110+0000","scrub_duration":0.010000867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":638,"num_write":117,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14952,"num_omap_keys":28,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"54'643","reported_seq":3598,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.506760+0000","last_change":"2024-08-18T13:43:56.143097+0000","last_active":"2024-08-18T13:51:49.506760+0000","last_peered":"2024-08-18T13:51:49.506760+0000","last_clean":"2024-08-18T13:51:49.506760+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:51:49.506760+0000","last_undegraded":"2024-08-18T13:51:49.506760+0000","last_fullsized":"2024-08-18T13:51:49.506760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079242040000000007,"stat_sum":{"num_bytes":41790406,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":892,"num_read_kb":23319,"num_write":639,"num_write_kb":45248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'82","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:12.049564+0000","last_change":"2024-08-18T13:52:12.049490+0000","last_active":"2024-08-18T13:52:12.049564+0000","last_peered":"2024-08-18T13:52:12.049564+0000","last_clean":"2024-08-18T13:52:12.049564+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:52:12.049564+0000","last_undegraded":"2024-08-18T13:52:12.049564+0000","last_fullsized":"2024-08-18T13:52:12.049564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'82","last_scrub_stamp":"2024-08-18T13:52:12.049277+0000","last_deep_scrub":"54'82","last_deep_scrub_stamp":"2024-08-18T13:52:12.049277+0000","last_clean_scrub_stamp":"2024-08-18T13:52:12.049277+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:42:58.521893+0000","scrub_duration":0.0038715189999999999,"objects_trimmed":0,"snaptrim_duration":0.0041495459999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"54'240","reported_seq":307,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:13.957043+0000","last_change":"2024-08-18T13:52:13.956993+0000","last_active":"2024-08-18T13:52:13.957043+0000","last_peered":"2024-08-18T13:52:13.957043+0000","last_clean":"2024-08-18T13:52:13.957043+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:52:13.957043+0000","last_undegraded":"2024-08-18T13:52:13.957043+0000","last_fullsized":"2024-08-18T13:52:13.957043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'240","last_scrub_stamp":"2024-08-18T13:52:13.956898+0000","last_deep_scrub":"54'240","last_deep_scrub_stamp":"2024-08-18T13:52:13.956898+0000","last_clean_scrub_stamp":"2024-08-18T13:52:13.956898+0000","objects_scrubbed":6,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:30:29.246162+0000","scrub_duration":0.0255698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":49152,"num_write":231,"num_write_kb":12421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24290,"num_omap_keys":46,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"54'716","reported_seq":3217,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.657979+0000","last_change":"2024-08-18T13:43:56.153929+0000","last_active":"2024-08-18T13:51:49.657979+0000","last_peered":"2024-08-18T13:51:49.657979+0000","last_clean":"2024-08-18T13:51:49.657979+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:51:49.657979+0000","last_undegraded":"2024-08-18T13:51:49.657979+0000","last_fullsized":"2024-08-18T13:51:49.657979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058974379999999996,"stat_sum":{"num_bytes":30462084,"num_objects":352,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":439,"num_read_kb":15189,"num_write":710,"num_write_kb":30067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"54'90","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.918324+0000","last_change":"2024-08-18T13:52:09.918219+0000","last_active":"2024-08-18T13:52:09.918324+0000","last_peered":"2024-08-18T13:52:09.918324+0000","last_clean":"2024-08-18T13:52:09.918324+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:52:09.918324+0000","last_undegraded":"2024-08-18T13:52:09.918324+0000","last_fullsized":"2024-08-18T13:52:09.918324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:09.917892+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:09.917892+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.917892+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:20.972039+0000","scrub_duration":0.0039382109999999996,"objects_trimmed":0,"snaptrim_duration":0.0068182370000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"54'983","reported_seq":1039,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.478599+0000","last_change":"2024-08-18T13:52:07.478553+0000","last_active":"2024-08-18T13:52:07.478599+0000","last_peered":"2024-08-18T13:52:07.478599+0000","last_clean":"2024-08-18T13:52:07.478599+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:52:07.478599+0000","last_undegraded":"2024-08-18T13:52:07.478599+0000","last_fullsized":"2024-08-18T13:52:07.478599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'983","last_scrub_stamp":"2024-08-18T13:52:07.478392+0000","last_deep_scrub":"54'983","last_deep_scrub_stamp":"2024-08-18T13:52:07.478392+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.478392+0000","objects_scrubbed":2,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:52:04.274947+0000","scrub_duration":0.015274658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2087,"num_write":979,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8826,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"54'119","reported_seq":208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:06.906625+0000","last_change":"2024-08-18T13:52:06.906561+0000","last_active":"2024-08-18T13:52:06.906625+0000","last_peered":"2024-08-18T13:52:06.906625+0000","last_clean":"2024-08-18T13:52:06.906625+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:52:06.906625+0000","last_undegraded":"2024-08-18T13:52:06.906625+0000","last_fullsized":"2024-08-18T13:52:06.906625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'119","last_scrub_stamp":"2024-08-18T13:52:06.906373+0000","last_deep_scrub":"54'119","last_deep_scrub_stamp":"2024-08-18T13:52:06.906373+0000","last_clean_scrub_stamp":"2024-08-18T13:52:06.906373+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:57:50.221813+0000","scrub_duration":0.0082944209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"54'664","reported_seq":3525,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.653284+0000","last_change":"2024-08-18T13:43:55.956673+0000","last_active":"2024-08-18T13:51:49.653284+0000","last_peered":"2024-08-18T13:51:49.653284+0000","last_clean":"2024-08-18T13:51:49.653284+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:51:49.653284+0000","last_undegraded":"2024-08-18T13:51:49.653284+0000","last_fullsized":"2024-08-18T13:51:49.653284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035684480000000001,"stat_sum":{"num_bytes":39930170,"num_objects":364,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":798,"num_read_kb":26403,"num_write":662,"num_write_kb":47533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"54'92","reported_seq":2156,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781612+0000","last_change":"2024-08-18T13:43:55.552313+0000","last_active":"2024-08-18T13:51:51.781612+0000","last_peered":"2024-08-18T13:51:51.781612+0000","last_clean":"2024-08-18T13:51:51.781612+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:51:51.781612+0000","last_undegraded":"2024-08-18T13:51:51.781612+0000","last_fullsized":"2024-08-18T13:51:51.781612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057711639999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"54'80","reported_seq":133,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.495618+0000","last_change":"2024-08-18T13:52:16.495514+0000","last_active":"2024-08-18T13:52:16.495618+0000","last_peered":"2024-08-18T13:52:16.495618+0000","last_clean":"2024-08-18T13:52:16.495618+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:52:16.495618+0000","last_undegraded":"2024-08-18T13:52:16.495618+0000","last_fullsized":"2024-08-18T13:52:16.495618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'80","last_scrub_stamp":"2024-08-18T13:52:16.495259+0000","last_deep_scrub":"54'80","last_deep_scrub_stamp":"2024-08-18T13:52:16.495259+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.495259+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:47:37.229120+0000","scrub_duration":0.016550458000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31498,"num_omap_keys":60,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"54'688","reported_seq":4128,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.589132+0000","last_change":"2024-08-18T13:43:55.953251+0000","last_active":"2024-08-18T13:51:49.589132+0000","last_peered":"2024-08-18T13:51:49.589132+0000","last_clean":"2024-08-18T13:51:49.589132+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:51:49.589132+0000","last_undegraded":"2024-08-18T13:51:49.589132+0000","last_fullsized":"2024-08-18T13:51:49.589132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032433420000000002,"stat_sum":{"num_bytes":54557211,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1377,"num_read_kb":50055,"num_write":687,"num_write_kb":57734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"54'95","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.756393+0000","last_change":"2024-08-18T13:43:55.918416+0000","last_active":"2024-08-18T13:51:51.756393+0000","last_peered":"2024-08-18T13:51:51.756393+0000","last_clean":"2024-08-18T13:51:51.756393+0000","last_became_active":"2024-08-18T13:40:16.614814+0000","last_became_peered":"2024-08-18T13:40:16.614814+0000","last_unstale":"2024-08-18T13:51:51.756393+0000","last_undegraded":"2024-08-18T13:51:51.756393+0000","last_fullsized":"2024-08-18T13:51:51.756393+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033975450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'36","reported_seq":101,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.170255+0000","last_change":"2024-08-18T13:52:10.170197+0000","last_active":"2024-08-18T13:52:10.170255+0000","last_peered":"2024-08-18T13:52:10.170255+0000","last_clean":"2024-08-18T13:52:10.170255+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:52:10.170255+0000","last_undegraded":"2024-08-18T13:52:10.170255+0000","last_fullsized":"2024-08-18T13:52:10.170255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'36","last_scrub_stamp":"2024-08-18T13:52:10.170013+0000","last_deep_scrub":"54'36","last_deep_scrub_stamp":"2024-08-18T13:52:10.170013+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.170013+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:21.224160+0000","scrub_duration":0.017405436999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":33,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24938,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"54'670","reported_seq":3235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.683385+0000","last_change":"2024-08-18T13:43:56.153281+0000","last_active":"2024-08-18T13:51:49.683385+0000","last_peered":"2024-08-18T13:51:49.683385+0000","last_clean":"2024-08-18T13:51:49.683385+0000","last_became_active":"2024-08-18T13:40:16.609908+0000","last_became_peered":"2024-08-18T13:40:16.609908+0000","last_unstale":"2024-08-18T13:51:49.683385+0000","last_undegraded":"2024-08-18T13:51:49.683385+0000","last_fullsized":"2024-08-18T13:51:49.683385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069871079999999997,"stat_sum":{"num_bytes":26719276,"num_objects":358,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":491,"num_read_kb":18148,"num_write":671,"num_write_kb":26414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4,5],"acting":[0,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'87","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784671+0000","last_change":"2024-08-18T13:43:55.553120+0000","last_active":"2024-08-18T13:51:51.784671+0000","last_peered":"2024-08-18T13:51:51.784671+0000","last_clean":"2024-08-18T13:51:51.784671+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:51:51.784671+0000","last_undegraded":"2024-08-18T13:51:51.784671+0000","last_fullsized":"2024-08-18T13:51:51.784671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057951929999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"54'32","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.525558+0000","last_change":"2024-08-18T13:52:18.525490+0000","last_active":"2024-08-18T13:52:18.525558+0000","last_peered":"2024-08-18T13:52:18.525558+0000","last_clean":"2024-08-18T13:52:18.525558+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:52:18.525558+0000","last_undegraded":"2024-08-18T13:52:18.525558+0000","last_fullsized":"2024-08-18T13:52:18.525558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'32","last_scrub_stamp":"2024-08-18T13:52:18.525224+0000","last_deep_scrub":"54'32","last_deep_scrub_stamp":"2024-08-18T13:52:18.525224+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.525224+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:52:15.321780+0000","scrub_duration":0.016102814999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":30,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9612,"num_omap_keys":18,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"54'688","reported_seq":2819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.691168+0000","last_change":"2024-08-18T13:43:55.859666+0000","last_active":"2024-08-18T13:51:49.691168+0000","last_peered":"2024-08-18T13:51:49.691168+0000","last_clean":"2024-08-18T13:51:49.691168+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:51:49.691168+0000","last_undegraded":"2024-08-18T13:51:49.691168+0000","last_fullsized":"2024-08-18T13:51:49.691168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070733389999999997,"stat_sum":{"num_bytes":17282460,"num_objects":346,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":68,"num_read_kb":8224,"num_write":686,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"54'66","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.910326+0000","last_change":"2024-08-18T13:52:04.910259+0000","last_active":"2024-08-18T13:52:04.910326+0000","last_peered":"2024-08-18T13:52:04.910326+0000","last_clean":"2024-08-18T13:52:04.910326+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:52:04.910326+0000","last_undegraded":"2024-08-18T13:52:04.910326+0000","last_fullsized":"2024-08-18T13:52:04.910326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'66","last_scrub_stamp":"2024-08-18T13:52:04.910054+0000","last_deep_scrub":"54'66","last_deep_scrub_stamp":"2024-08-18T13:52:04.910054+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.910054+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:14:14.576424+0000","scrub_duration":0.0032501320000000002,"objects_trimmed":0,"snaptrim_duration":0.0060906930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"54'29","reported_seq":94,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.600601+0000","last_change":"2024-08-18T13:52:04.600549+0000","last_active":"2024-08-18T13:52:04.600601+0000","last_peered":"2024-08-18T13:52:04.600601+0000","last_clean":"2024-08-18T13:52:04.600601+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:52:04.600601+0000","last_undegraded":"2024-08-18T13:52:04.600601+0000","last_fullsized":"2024-08-18T13:52:04.600601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'29","last_scrub_stamp":"2024-08-18T13:52:04.600439+0000","last_deep_scrub":"54'29","last_deep_scrub_stamp":"2024-08-18T13:52:04.600439+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.600439+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:26.017443+0000","scrub_duration":0.014861572999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":27,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"54'695","reported_seq":3631,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.700276+0000","last_change":"2024-08-18T13:43:55.557389+0000","last_active":"2024-08-18T13:51:49.700276+0000","last_peered":"2024-08-18T13:51:49.700276+0000","last_clean":"2024-08-18T13:51:49.700276+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:51:49.700276+0000","last_undegraded":"2024-08-18T13:51:49.700276+0000","last_fullsized":"2024-08-18T13:51:49.700276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011606655,"stat_sum":{"num_bytes":44119789,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":873,"num_read_kb":37157,"num_write":696,"num_write_kb":47501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"54'91","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784133+0000","last_change":"2024-08-18T13:43:55.553915+0000","last_active":"2024-08-18T13:51:51.784133+0000","last_peered":"2024-08-18T13:51:51.784133+0000","last_clean":"2024-08-18T13:51:51.784133+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:51:51.784133+0000","last_undegraded":"2024-08-18T13:51:51.784133+0000","last_fullsized":"2024-08-18T13:51:51.784133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053019490000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"54'1212","reported_seq":1295,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.493306+0000","last_change":"2024-08-18T13:52:14.493275+0000","last_active":"2024-08-18T13:52:14.493306+0000","last_peered":"2024-08-18T13:52:14.493306+0000","last_clean":"2024-08-18T13:52:14.493306+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:52:14.493306+0000","last_undegraded":"2024-08-18T13:52:14.493306+0000","last_fullsized":"2024-08-18T13:52:14.493306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'1212","last_scrub_stamp":"2024-08-18T13:52:14.493191+0000","last_deep_scrub":"54'1212","last_deep_scrub_stamp":"2024-08-18T13:52:14.493191+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.493191+0000","objects_scrubbed":8,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:29:04.116014+0000","scrub_duration":0.026335695999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":78,"num_read_kb":12399,"num_write":1202,"num_write_kb":8830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18540,"num_omap_keys":36,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'657","reported_seq":3517,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.698516+0000","last_change":"2024-08-18T13:43:56.154837+0000","last_active":"2024-08-18T13:51:49.698516+0000","last_peered":"2024-08-18T13:51:49.698516+0000","last_clean":"2024-08-18T13:51:49.698516+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:51:49.698516+0000","last_undegraded":"2024-08-18T13:51:49.698516+0000","last_fullsized":"2024-08-18T13:51:49.698516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054104280000000001,"stat_sum":{"num_bytes":33280018,"num_objects":296,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296,"num_whiteouts":0,"num_read":797,"num_read_kb":22024,"num_write":655,"num_write_kb":32783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.315208+0000","last_change":"2024-08-18T13:52:18.315124+0000","last_active":"2024-08-18T13:52:18.315208+0000","last_peered":"2024-08-18T13:52:18.315208+0000","last_clean":"2024-08-18T13:52:18.315208+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:52:18.315208+0000","last_undegraded":"2024-08-18T13:52:18.315208+0000","last_fullsized":"2024-08-18T13:52:18.315208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:18.314906+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:18.314906+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.314906+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:39.731910+0000","scrub_duration":0.0038296580000000001,"objects_trimmed":0,"snaptrim_duration":0.0044255090000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"54'8","reported_seq":61,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.301096+0000","last_change":"2024-08-18T13:52:09.301035+0000","last_active":"2024-08-18T13:52:09.301096+0000","last_peered":"2024-08-18T13:52:09.301096+0000","last_clean":"2024-08-18T13:52:09.301096+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:52:09.301096+0000","last_undegraded":"2024-08-18T13:52:09.301096+0000","last_fullsized":"2024-08-18T13:52:09.301096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'8","last_scrub_stamp":"2024-08-18T13:52:09.300822+0000","last_deep_scrub":"54'8","last_deep_scrub_stamp":"2024-08-18T13:52:09.300822+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.300822+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:39:45.416911+0000","scrub_duration":0.008909953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28532,"num_omap_keys":54,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"54'660","reported_seq":2780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.749372+0000","last_change":"2024-08-18T13:43:55.555242+0000","last_active":"2024-08-18T13:51:49.749372+0000","last_peered":"2024-08-18T13:51:49.749372+0000","last_clean":"2024-08-18T13:51:49.749372+0000","last_became_active":"2024-08-18T13:40:16.915369+0000","last_became_peered":"2024-08-18T13:40:16.915369+0000","last_unstale":"2024-08-18T13:51:49.749372+0000","last_undegraded":"2024-08-18T13:51:49.749372+0000","last_fullsized":"2024-08-18T13:51:49.749372+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005619059,"stat_sum":{"num_bytes":16977484,"num_objects":353,"num_object_clones":0,"num_object_copies":1412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":46,"num_read_kb":4277,"num_write":660,"num_write_kb":20975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,6],"acting":[3,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"54'101","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.784740+0000","last_change":"2024-08-18T13:43:55.551514+0000","last_active":"2024-08-18T13:51:51.784740+0000","last_peered":"2024-08-18T13:51:51.784740+0000","last_clean":"2024-08-18T13:51:51.784740+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:51:51.784740+0000","last_undegraded":"2024-08-18T13:51:51.784740+0000","last_fullsized":"2024-08-18T13:51:51.784740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064019180000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"54'4","reported_seq":58,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.395883+0000","last_change":"2024-08-18T13:52:15.395814+0000","last_active":"2024-08-18T13:52:15.395883+0000","last_peered":"2024-08-18T13:52:15.395883+0000","last_clean":"2024-08-18T13:52:15.395883+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:52:15.395883+0000","last_undegraded":"2024-08-18T13:52:15.395883+0000","last_fullsized":"2024-08-18T13:52:15.395883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'4","last_scrub_stamp":"2024-08-18T13:52:15.395609+0000","last_deep_scrub":"54'4","last_deep_scrub_stamp":"2024-08-18T13:52:15.395609+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.395609+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:46.408855+0000","scrub_duration":0.0073810990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15112,"num_omap_keys":29,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"54'625","reported_seq":3599,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.714049+0000","last_change":"2024-08-18T13:43:55.554811+0000","last_active":"2024-08-18T13:51:49.714049+0000","last_peered":"2024-08-18T13:51:49.714049+0000","last_clean":"2024-08-18T13:51:49.714049+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:51:49.714049+0000","last_undegraded":"2024-08-18T13:51:49.714049+0000","last_fullsized":"2024-08-18T13:51:49.714049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074125900000000002,"stat_sum":{"num_bytes":40693231,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":911,"num_read_kb":37155,"num_write":622,"num_write_kb":48255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.056426+0000","last_change":"2024-08-18T13:52:08.056345+0000","last_active":"2024-08-18T13:52:08.056426+0000","last_peered":"2024-08-18T13:52:08.056426+0000","last_clean":"2024-08-18T13:52:08.056426+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:52:08.056426+0000","last_undegraded":"2024-08-18T13:52:08.056426+0000","last_fullsized":"2024-08-18T13:52:08.056426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:08.056130+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:08.056130+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.056130+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:54.094989+0000","scrub_duration":0.003803248,"objects_trimmed":0,"snaptrim_duration":0.0059765870000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"54'220","reported_seq":298,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:02.096320+0000","last_change":"2024-08-18T13:52:02.096276+0000","last_active":"2024-08-18T13:52:02.096320+0000","last_peered":"2024-08-18T13:52:02.096320+0000","last_clean":"2024-08-18T13:52:02.096320+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:52:02.096320+0000","last_undegraded":"2024-08-18T13:52:02.096320+0000","last_fullsized":"2024-08-18T13:52:02.096320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'220","last_scrub_stamp":"2024-08-18T13:52:02.096123+0000","last_deep_scrub":"54'220","last_deep_scrub_stamp":"2024-08-18T13:52:02.096123+0000","last_clean_scrub_stamp":"2024-08-18T13:52:02.096123+0000","objects_scrubbed":6,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:09.274537+0000","scrub_duration":0.023746768000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":45098,"num_write":215,"num_write_kb":8331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11822,"num_omap_keys":23,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'629","reported_seq":3942,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.559941+0000","last_change":"2024-08-18T13:43:55.862627+0000","last_active":"2024-08-18T13:51:49.559941+0000","last_peered":"2024-08-18T13:51:49.559941+0000","last_clean":"2024-08-18T13:51:49.559941+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:51:49.559941+0000","last_undegraded":"2024-08-18T13:51:49.559941+0000","last_fullsized":"2024-08-18T13:51:49.559941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084356780000000003,"stat_sum":{"num_bytes":50488256,"num_objects":306,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":1250,"num_read_kb":43248,"num_write":629,"num_write_kb":49603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'84","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781677+0000","last_change":"2024-08-18T13:43:55.551442+0000","last_active":"2024-08-18T13:51:51.781677+0000","last_peered":"2024-08-18T13:51:51.781677+0000","last_clean":"2024-08-18T13:51:51.781677+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:51:51.781677+0000","last_undegraded":"2024-08-18T13:51:51.781677+0000","last_fullsized":"2024-08-18T13:51:51.781677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065977270000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"54'241","reported_seq":325,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:12.244378+0000","last_change":"2024-08-18T13:52:12.244303+0000","last_active":"2024-08-18T13:52:12.244378+0000","last_peered":"2024-08-18T13:52:12.244378+0000","last_clean":"2024-08-18T13:52:12.244378+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:52:12.244378+0000","last_undegraded":"2024-08-18T13:52:12.244378+0000","last_fullsized":"2024-08-18T13:52:12.244378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'240","last_scrub_stamp":"2024-08-18T13:52:12.244185+0000","last_deep_scrub":"54'240","last_deep_scrub_stamp":"2024-08-18T13:52:12.244185+0000","last_clean_scrub_stamp":"2024-08-18T13:52:12.244185+0000","objects_scrubbed":8,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:30:27.533449+0000","scrub_duration":0.029778697999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583572,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":53278,"num_write":227,"num_write_kb":12433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22962,"num_omap_keys":43,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"54'671","reported_seq":3263,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.572954+0000","last_change":"2024-08-18T13:43:56.142175+0000","last_active":"2024-08-18T13:51:49.572954+0000","last_peered":"2024-08-18T13:51:49.572954+0000","last_clean":"2024-08-18T13:51:49.572954+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:51:49.572954+0000","last_undegraded":"2024-08-18T13:51:49.572954+0000","last_fullsized":"2024-08-18T13:51:49.572954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076714670000000004,"stat_sum":{"num_bytes":16086174,"num_objects":343,"num_object_clones":0,"num_object_copies":1372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":529,"num_read_kb":12373,"num_write":668,"num_write_kb":24206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"54'86","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.670616+0000","last_change":"2024-08-18T13:52:15.670550+0000","last_active":"2024-08-18T13:52:15.670616+0000","last_peered":"2024-08-18T13:52:15.670616+0000","last_clean":"2024-08-18T13:52:15.670616+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:52:15.670616+0000","last_undegraded":"2024-08-18T13:52:15.670616+0000","last_fullsized":"2024-08-18T13:52:15.670616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'86","last_scrub_stamp":"2024-08-18T13:52:15.670343+0000","last_deep_scrub":"54'86","last_deep_scrub_stamp":"2024-08-18T13:52:15.670343+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.670343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:49.587682+0000","scrub_duration":0.0034242740000000002,"objects_trimmed":0,"snaptrim_duration":0.0039212459999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"54'173","reported_seq":242,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.045210+0000","last_change":"2024-08-18T13:52:14.045155+0000","last_active":"2024-08-18T13:52:14.045210+0000","last_peered":"2024-08-18T13:52:14.045210+0000","last_clean":"2024-08-18T13:52:14.045210+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:52:14.045210+0000","last_undegraded":"2024-08-18T13:52:14.045210+0000","last_fullsized":"2024-08-18T13:52:14.045210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'173","last_scrub_stamp":"2024-08-18T13:52:14.044993+0000","last_deep_scrub":"54'173","last_deep_scrub_stamp":"2024-08-18T13:52:14.044993+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.044993+0000","objects_scrubbed":4,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:39.376618+0000","scrub_duration":0.013861017999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8216,"num_write":168,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'644","reported_seq":3279,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.705949+0000","last_change":"2024-08-18T13:43:55.559022+0000","last_active":"2024-08-18T13:51:49.705949+0000","last_peered":"2024-08-18T13:51:49.705949+0000","last_clean":"2024-08-18T13:51:49.705949+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:51:49.705949+0000","last_undegraded":"2024-08-18T13:51:49.705949+0000","last_fullsized":"2024-08-18T13:51:49.705949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059375670000000004,"stat_sum":{"num_bytes":15740586,"num_objects":325,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":572,"num_read_kb":18359,"num_write":643,"num_write_kb":23895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"54'89","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.002384+0000","last_change":"2024-08-18T13:52:14.002238+0000","last_active":"2024-08-18T13:52:14.002384+0000","last_peered":"2024-08-18T13:52:14.002384+0000","last_clean":"2024-08-18T13:52:14.002384+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:52:14.002384+0000","last_undegraded":"2024-08-18T13:52:14.002384+0000","last_fullsized":"2024-08-18T13:52:14.002384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'89","last_scrub_stamp":"2024-08-18T13:52:14.002012+0000","last_deep_scrub":"54'89","last_deep_scrub_stamp":"2024-08-18T13:52:14.002012+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.002012+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:43.739908+0000","scrub_duration":0.0036712979999999999,"objects_trimmed":0,"snaptrim_duration":0.0057226309999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"54'237","reported_seq":297,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.513342+0000","last_change":"2024-08-18T13:52:05.513291+0000","last_active":"2024-08-18T13:52:05.513342+0000","last_peered":"2024-08-18T13:52:05.513342+0000","last_clean":"2024-08-18T13:52:05.513342+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:52:05.513342+0000","last_undegraded":"2024-08-18T13:52:05.513342+0000","last_fullsized":"2024-08-18T13:52:05.513342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'237","last_scrub_stamp":"2024-08-18T13:52:05.513154+0000","last_deep_scrub":"54'237","last_deep_scrub_stamp":"2024-08-18T13:52:05.513154+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.513154+0000","objects_scrubbed":4,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:47:26.247014+0000","scrub_duration":0.018778702000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":20397,"num_write":232,"num_write_kb":4254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17234,"num_omap_keys":33,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"54'677","reported_seq":3170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.525636+0000","last_change":"2024-08-18T13:43:55.553346+0000","last_active":"2024-08-18T13:51:49.525636+0000","last_peered":"2024-08-18T13:51:49.525636+0000","last_clean":"2024-08-18T13:51:49.525636+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:51:49.525636+0000","last_undegraded":"2024-08-18T13:51:49.525636+0000","last_fullsized":"2024-08-18T13:51:49.525636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073620359999999998,"stat_sum":{"num_bytes":31240743,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":431,"num_read_kb":13577,"num_write":670,"num_write_kb":34133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"54'106","reported_seq":2181,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.455857+0000","last_change":"2024-08-18T13:52:18.455745+0000","last_active":"2024-08-18T13:52:18.455857+0000","last_peered":"2024-08-18T13:52:18.455857+0000","last_clean":"2024-08-18T13:52:18.455857+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:52:18.455857+0000","last_undegraded":"2024-08-18T13:52:18.455857+0000","last_fullsized":"2024-08-18T13:52:18.455857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'106","last_scrub_stamp":"2024-08-18T13:52:18.455540+0000","last_deep_scrub":"54'106","last_deep_scrub_stamp":"2024-08-18T13:52:18.455540+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.455540+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:15.305313+0000","scrub_duration":0.003717709,"objects_trimmed":0,"snaptrim_duration":0.0059613369999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"54'14","reported_seq":67,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.267260+0000","last_change":"2024-08-18T13:52:14.267230+0000","last_active":"2024-08-18T13:52:14.267260+0000","last_peered":"2024-08-18T13:52:14.267260+0000","last_clean":"2024-08-18T13:52:14.267260+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:52:14.267260+0000","last_undegraded":"2024-08-18T13:52:14.267260+0000","last_fullsized":"2024-08-18T13:52:14.267260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'14","last_scrub_stamp":"2024-08-18T13:52:14.267137+0000","last_deep_scrub":"54'14","last_deep_scrub_stamp":"2024-08-18T13:52:14.267137+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.267137+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:59.683518+0000","scrub_duration":0.010501551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":39082,"num_omap_keys":74,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"54'699","reported_seq":3481,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.677703+0000","last_change":"2024-08-18T13:43:55.553902+0000","last_active":"2024-08-18T13:51:49.677703+0000","last_peered":"2024-08-18T13:51:49.677703+0000","last_clean":"2024-08-18T13:51:49.677703+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:51:49.677703+0000","last_undegraded":"2024-08-18T13:51:49.677703+0000","last_fullsized":"2024-08-18T13:51:49.677703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076808309999999999,"stat_sum":{"num_bytes":29636158,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":719,"num_read_kb":24704,"num_write":698,"num_write_kb":38783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"54'650","reported_seq":3293,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.152118+0000","last_change":"2024-08-18T13:43:56.152924+0000","last_active":"2024-08-18T13:51:49.152118+0000","last_peered":"2024-08-18T13:51:49.152118+0000","last_clean":"2024-08-18T13:51:49.152118+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:51:49.152118+0000","last_undegraded":"2024-08-18T13:51:49.152118+0000","last_fullsized":"2024-08-18T13:51:49.152118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068864499999999997,"stat_sum":{"num_bytes":24960867,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":580,"num_read_kb":20952,"num_write":650,"num_write_kb":24715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"54'91","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.205937+0000","last_change":"2024-08-18T13:52:05.205870+0000","last_active":"2024-08-18T13:52:05.205937+0000","last_peered":"2024-08-18T13:52:05.205937+0000","last_clean":"2024-08-18T13:52:05.205937+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:52:05.205937+0000","last_undegraded":"2024-08-18T13:52:05.205937+0000","last_fullsized":"2024-08-18T13:52:05.205937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:05.205644+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:05.205644+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.205644+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:51.244504+0000","scrub_duration":0.0033367140000000002,"objects_trimmed":0,"snaptrim_duration":0.0066095759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"54'203","reported_seq":267,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.395398+0000","last_change":"2024-08-18T13:52:10.395341+0000","last_active":"2024-08-18T13:52:10.395398+0000","last_peered":"2024-08-18T13:52:10.395398+0000","last_clean":"2024-08-18T13:52:10.395398+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:52:10.395398+0000","last_undegraded":"2024-08-18T13:52:10.395398+0000","last_fullsized":"2024-08-18T13:52:10.395398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'203","last_scrub_stamp":"2024-08-18T13:52:10.395155+0000","last_deep_scrub":"54'203","last_deep_scrub_stamp":"2024-08-18T13:52:10.395155+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.395155+0000","objects_scrubbed":6,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:17.573569+0000","scrub_duration":0.030766387999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":197,"num_write_kb":12441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25166,"num_omap_keys":49,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"54'658","reported_seq":3188,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.719087+0000","last_change":"2024-08-18T13:43:55.958984+0000","last_active":"2024-08-18T13:51:49.719087+0000","last_peered":"2024-08-18T13:51:49.719087+0000","last_clean":"2024-08-18T13:51:49.719087+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:51:49.719087+0000","last_undegraded":"2024-08-18T13:51:49.719087+0000","last_fullsized":"2024-08-18T13:51:49.719087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047613839999999996,"stat_sum":{"num_bytes":20867535,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":467,"num_read_kb":16390,"num_write":656,"num_write_kb":20726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":2173,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.765271+0000","last_change":"2024-08-18T13:43:55.552907+0000","last_active":"2024-08-18T13:51:51.765271+0000","last_peered":"2024-08-18T13:51:51.765271+0000","last_clean":"2024-08-18T13:51:51.765271+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:51:51.765271+0000","last_undegraded":"2024-08-18T13:51:51.765271+0000","last_fullsized":"2024-08-18T13:51:51.765271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068339799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"54'105","reported_seq":163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.093354+0000","last_change":"2024-08-18T13:52:21.093276+0000","last_active":"2024-08-18T13:52:21.093354+0000","last_peered":"2024-08-18T13:52:21.093354+0000","last_clean":"2024-08-18T13:52:21.093354+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:52:21.093354+0000","last_undegraded":"2024-08-18T13:52:21.093354+0000","last_fullsized":"2024-08-18T13:52:21.093354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'105","last_scrub_stamp":"2024-08-18T13:52:21.093147+0000","last_deep_scrub":"54'105","last_deep_scrub_stamp":"2024-08-18T13:52:21.093147+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.093147+0000","objects_scrubbed":2,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:53:06.284734+0000","scrub_duration":0.015701764999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":103,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7766,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"54'135","reported_seq":201,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.254544+0000","last_change":"2024-08-18T13:52:04.254482+0000","last_active":"2024-08-18T13:52:04.254544+0000","last_peered":"2024-08-18T13:52:04.254544+0000","last_clean":"2024-08-18T13:52:04.254544+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:52:04.254544+0000","last_undegraded":"2024-08-18T13:52:04.254544+0000","last_fullsized":"2024-08-18T13:52:04.254544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'135","last_scrub_stamp":"2024-08-18T13:52:04.254271+0000","last_deep_scrub":"54'135","last_deep_scrub_stamp":"2024-08-18T13:52:04.254271+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.254271+0000","objects_scrubbed":5,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:13:13.411254+0000","scrub_duration":0.03253764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":128,"num_write_kb":12417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24866,"num_omap_keys":48,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"54'91","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.023472+0000","last_change":"2024-08-18T13:52:10.023405+0000","last_active":"2024-08-18T13:52:10.023472+0000","last_peered":"2024-08-18T13:52:10.023472+0000","last_clean":"2024-08-18T13:52:10.023472+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:52:10.023472+0000","last_undegraded":"2024-08-18T13:52:10.023472+0000","last_fullsized":"2024-08-18T13:52:10.023472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:10.023201+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:10.023201+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.023201+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:06.872974+0000","scrub_duration":0.003062024,"objects_trimmed":0,"snaptrim_duration":0.0053680339999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"54'664","reported_seq":3635,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.695499+0000","last_change":"2024-08-18T13:43:55.917879+0000","last_active":"2024-08-18T13:51:49.695499+0000","last_peered":"2024-08-18T13:51:49.695499+0000","last_clean":"2024-08-18T13:51:49.695499+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:51:49.695499+0000","last_undegraded":"2024-08-18T13:51:49.695499+0000","last_fullsized":"2024-08-18T13:51:49.695499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035122529999999999,"stat_sum":{"num_bytes":30328916,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":908,"num_read_kb":29115,"num_write":663,"num_write_kb":34071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"54'17","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.959979+0000","last_change":"2024-08-18T13:52:10.959908+0000","last_active":"2024-08-18T13:52:10.959979+0000","last_peered":"2024-08-18T13:52:10.959979+0000","last_clean":"2024-08-18T13:52:10.959979+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:52:10.959979+0000","last_undegraded":"2024-08-18T13:52:10.959979+0000","last_fullsized":"2024-08-18T13:52:10.959979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'17","last_scrub_stamp":"2024-08-18T13:52:10.959683+0000","last_deep_scrub":"54'17","last_deep_scrub_stamp":"2024-08-18T13:52:10.959683+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.959683+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:40.697579+0000","scrub_duration":0.013613793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":19,"num_write_kb":4249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21984,"num_omap_keys":42,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.601341+0000","last_change":"2024-08-18T13:52:07.601203+0000","last_active":"2024-08-18T13:52:07.601341+0000","last_peered":"2024-08-18T13:52:07.601341+0000","last_clean":"2024-08-18T13:52:07.601341+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:52:07.601341+0000","last_undegraded":"2024-08-18T13:52:07.601341+0000","last_fullsized":"2024-08-18T13:52:07.601341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:07.600983+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:07.600983+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.600983+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:43:21.683007+0000","scrub_duration":0.0035012659999999998,"objects_trimmed":0,"snaptrim_duration":0.0072935329999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"54'656","reported_seq":3235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.705587+0000","last_change":"2024-08-18T13:43:55.555171+0000","last_active":"2024-08-18T13:51:49.705587+0000","last_peered":"2024-08-18T13:51:49.705587+0000","last_clean":"2024-08-18T13:51:49.705587+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:51:49.705587+0000","last_undegraded":"2024-08-18T13:51:49.705587+0000","last_fullsized":"2024-08-18T13:51:49.705587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063112100000000003,"stat_sum":{"num_bytes":21201155,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":516,"num_read_kb":24278,"num_write":657,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"54'107","reported_seq":2181,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.370471+0000","last_change":"2024-08-18T13:52:21.370406+0000","last_active":"2024-08-18T13:52:21.370471+0000","last_peered":"2024-08-18T13:52:21.370471+0000","last_clean":"2024-08-18T13:52:21.370471+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:52:21.370471+0000","last_undegraded":"2024-08-18T13:52:21.370471+0000","last_fullsized":"2024-08-18T13:52:21.370471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'107","last_scrub_stamp":"2024-08-18T13:52:21.370198+0000","last_deep_scrub":"54'107","last_deep_scrub_stamp":"2024-08-18T13:52:21.370198+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.370198+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:44:33.767918+0000","scrub_duration":0.003909615,"objects_trimmed":0,"snaptrim_duration":0.0045422170000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"54'110","reported_seq":2174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777047+0000","last_change":"2024-08-18T13:43:55.919976+0000","last_active":"2024-08-18T13:51:51.777047+0000","last_peered":"2024-08-18T13:51:51.777047+0000","last_clean":"2024-08-18T13:51:51.777047+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:51:51.777047+0000","last_undegraded":"2024-08-18T13:51:51.777047+0000","last_fullsized":"2024-08-18T13:51:51.777047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032950929999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"54'101","reported_seq":2185,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:17.440214+0000","last_change":"2024-08-18T13:52:17.440127+0000","last_active":"2024-08-18T13:52:17.440214+0000","last_peered":"2024-08-18T13:52:17.440214+0000","last_clean":"2024-08-18T13:52:17.440214+0000","last_became_active":"2024-08-18T13:40:16.610568+0000","last_became_peered":"2024-08-18T13:40:16.610568+0000","last_unstale":"2024-08-18T13:52:17.440214+0000","last_undegraded":"2024-08-18T13:52:17.440214+0000","last_fullsized":"2024-08-18T13:52:17.440214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'101","last_scrub_stamp":"2024-08-18T13:52:17.439868+0000","last_deep_scrub":"54'101","last_deep_scrub_stamp":"2024-08-18T13:52:17.439868+0000","last_clean_scrub_stamp":"2024-08-18T13:52:17.439868+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:06:03.535494+0000","scrub_duration":0.003706352,"objects_trimmed":0,"snaptrim_duration":0.0066503589999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"54'99","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.766455+0000","last_change":"2024-08-18T13:43:56.153594+0000","last_active":"2024-08-18T13:51:51.766455+0000","last_peered":"2024-08-18T13:51:51.766455+0000","last_clean":"2024-08-18T13:51:51.766455+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:51:51.766455+0000","last_undegraded":"2024-08-18T13:51:51.766455+0000","last_fullsized":"2024-08-18T13:51:51.766455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056450420000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"54'80","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.895344+0000","last_change":"2024-08-18T13:52:05.895217+0000","last_active":"2024-08-18T13:52:05.895344+0000","last_peered":"2024-08-18T13:52:05.895344+0000","last_clean":"2024-08-18T13:52:05.895344+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:52:05.895344+0000","last_undegraded":"2024-08-18T13:52:05.895344+0000","last_fullsized":"2024-08-18T13:52:05.895344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'80","last_scrub_stamp":"2024-08-18T13:52:05.894985+0000","last_deep_scrub":"54'80","last_deep_scrub_stamp":"2024-08-18T13:52:05.894985+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.894985+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:51.990610+0000","scrub_duration":0.0040115990000000002,"objects_trimmed":0,"snaptrim_duration":0.0068042459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"54'74","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.251604+0000","last_change":"2024-08-18T13:52:04.251540+0000","last_active":"2024-08-18T13:52:04.251604+0000","last_peered":"2024-08-18T13:52:04.251604+0000","last_clean":"2024-08-18T13:52:04.251604+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:52:04.251604+0000","last_undegraded":"2024-08-18T13:52:04.251604+0000","last_fullsized":"2024-08-18T13:52:04.251604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'74","last_scrub_stamp":"2024-08-18T13:52:04.251343+0000","last_deep_scrub":"54'74","last_deep_scrub_stamp":"2024-08-18T13:52:04.251343+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.251343+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:35.264589+0000","scrub_duration":0.003478264,"objects_trimmed":0,"snaptrim_duration":0.0079157710000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"54'109","reported_seq":2173,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.779905+0000","last_change":"2024-08-18T13:43:55.959839+0000","last_active":"2024-08-18T13:51:51.779905+0000","last_peered":"2024-08-18T13:51:51.779905+0000","last_clean":"2024-08-18T13:51:51.779905+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:51:51.779905+0000","last_undegraded":"2024-08-18T13:51:51.779905+0000","last_fullsized":"2024-08-18T13:51:51.779905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029546540000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"54'85","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.755324+0000","last_change":"2024-08-18T13:43:55.555415+0000","last_active":"2024-08-18T13:51:51.755324+0000","last_peered":"2024-08-18T13:51:51.755324+0000","last_clean":"2024-08-18T13:51:51.755324+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:51:51.755324+0000","last_undegraded":"2024-08-18T13:51:51.755324+0000","last_fullsized":"2024-08-18T13:51:51.755324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055234810000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"54'92","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.762553+0000","last_change":"2024-08-18T13:43:55.916616+0000","last_active":"2024-08-18T13:51:51.762553+0000","last_peered":"2024-08-18T13:51:51.762553+0000","last_clean":"2024-08-18T13:51:51.762553+0000","last_became_active":"2024-08-18T13:40:16.614889+0000","last_became_peered":"2024-08-18T13:40:16.614889+0000","last_unstale":"2024-08-18T13:51:51.762553+0000","last_undegraded":"2024-08-18T13:51:51.762553+0000","last_fullsized":"2024-08-18T13:51:51.762553+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033397829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:03.919727+0000","last_change":"2024-08-18T13:52:03.919668+0000","last_active":"2024-08-18T13:52:03.919727+0000","last_peered":"2024-08-18T13:52:03.919727+0000","last_clean":"2024-08-18T13:52:03.919727+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:52:03.919727+0000","last_undegraded":"2024-08-18T13:52:03.919727+0000","last_fullsized":"2024-08-18T13:52:03.919727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:03.919468+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:03.919468+0000","last_clean_scrub_stamp":"2024-08-18T13:52:03.919468+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:34.932714+0000","scrub_duration":0.0037821840000000001,"objects_trimmed":0,"snaptrim_duration":0.0070000389999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"54'95","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.465675+0000","last_change":"2024-08-18T13:52:04.465574+0000","last_active":"2024-08-18T13:52:04.465675+0000","last_peered":"2024-08-18T13:52:04.465675+0000","last_clean":"2024-08-18T13:52:04.465675+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:52:04.465675+0000","last_undegraded":"2024-08-18T13:52:04.465675+0000","last_fullsized":"2024-08-18T13:52:04.465675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'95","last_scrub_stamp":"2024-08-18T13:52:04.465382+0000","last_deep_scrub":"54'95","last_deep_scrub_stamp":"2024-08-18T13:52:04.465382+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.465382+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:27.412242+0000","scrub_duration":0.003183946,"objects_trimmed":0,"snaptrim_duration":0.0070451519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"54'88","reported_seq":2153,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781337+0000","last_change":"2024-08-18T13:43:56.152879+0000","last_active":"2024-08-18T13:51:51.781337+0000","last_peered":"2024-08-18T13:51:51.781337+0000","last_clean":"2024-08-18T13:51:51.781337+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:51:51.781337+0000","last_undegraded":"2024-08-18T13:51:51.781337+0000","last_fullsized":"2024-08-18T13:51:51.781337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070254549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"54'98","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.760061+0000","last_change":"2024-08-18T13:43:55.956991+0000","last_active":"2024-08-18T13:51:51.760061+0000","last_peered":"2024-08-18T13:51:51.760061+0000","last_clean":"2024-08-18T13:51:51.760061+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:51:51.760061+0000","last_undegraded":"2024-08-18T13:51:51.760061+0000","last_fullsized":"2024-08-18T13:51:51.760061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003261548,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"54'85","reported_seq":2144,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758780+0000","last_change":"2024-08-18T13:43:56.141939+0000","last_active":"2024-08-18T13:51:51.758780+0000","last_peered":"2024-08-18T13:51:51.758780+0000","last_clean":"2024-08-18T13:51:51.758780+0000","last_became_active":"2024-08-18T13:40:16.612760+0000","last_became_peered":"2024-08-18T13:40:16.612760+0000","last_unstale":"2024-08-18T13:51:51.758780+0000","last_undegraded":"2024-08-18T13:51:51.758780+0000","last_fullsized":"2024-08-18T13:51:51.758780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076223990000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.547065+0000","last_change":"2024-08-18T13:52:20.546999+0000","last_active":"2024-08-18T13:52:20.547065+0000","last_peered":"2024-08-18T13:52:20.547065+0000","last_clean":"2024-08-18T13:52:20.547065+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:52:20.547065+0000","last_undegraded":"2024-08-18T13:52:20.547065+0000","last_fullsized":"2024-08-18T13:52:20.547065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:20.546797+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:20.546797+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.546797+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:47:09.544407+0000","scrub_duration":0.0042093260000000002,"objects_trimmed":0,"snaptrim_duration":0.0027557649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"54'94","reported_seq":2169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.412704+0000","last_change":"2024-08-18T13:52:16.412631+0000","last_active":"2024-08-18T13:52:16.412704+0000","last_peered":"2024-08-18T13:52:16.412704+0000","last_clean":"2024-08-18T13:52:16.412704+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:52:16.412704+0000","last_undegraded":"2024-08-18T13:52:16.412704+0000","last_fullsized":"2024-08-18T13:52:16.412704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'94","last_scrub_stamp":"2024-08-18T13:52:16.412429+0000","last_deep_scrub":"54'94","last_deep_scrub_stamp":"2024-08-18T13:52:16.412429+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.412429+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:02.451289+0000","scrub_duration":0.003524293,"objects_trimmed":0,"snaptrim_duration":0.0069473399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"54'96","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777169+0000","last_change":"2024-08-18T13:43:55.860127+0000","last_active":"2024-08-18T13:51:51.777169+0000","last_peered":"2024-08-18T13:51:51.777169+0000","last_clean":"2024-08-18T13:51:51.777169+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:51:51.777169+0000","last_undegraded":"2024-08-18T13:51:51.777169+0000","last_fullsized":"2024-08-18T13:51:51.777169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070196240000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"54'85","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.761280+0000","last_change":"2024-08-18T13:43:56.136762+0000","last_active":"2024-08-18T13:51:51.761280+0000","last_peered":"2024-08-18T13:51:51.761280+0000","last_clean":"2024-08-18T13:51:51.761280+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:51:51.761280+0000","last_undegraded":"2024-08-18T13:51:51.761280+0000","last_fullsized":"2024-08-18T13:51:51.761280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004509931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"54'92","reported_seq":2167,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.976679+0000","last_change":"2024-08-18T13:52:18.976615+0000","last_active":"2024-08-18T13:52:18.976679+0000","last_peered":"2024-08-18T13:52:18.976679+0000","last_clean":"2024-08-18T13:52:18.976679+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:52:18.976679+0000","last_undegraded":"2024-08-18T13:52:18.976679+0000","last_fullsized":"2024-08-18T13:52:18.976679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:18.976444+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:18.976444+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.976444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:27:33.305679+0000","scrub_duration":0.0033123919999999999,"objects_trimmed":0,"snaptrim_duration":0.0062343609999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"54'81","reported_seq":2145,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.767339+0000","last_change":"2024-08-18T13:43:55.554507+0000","last_active":"2024-08-18T13:51:51.767339+0000","last_peered":"2024-08-18T13:51:51.767339+0000","last_clean":"2024-08-18T13:51:51.767339+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:51:51.767339+0000","last_undegraded":"2024-08-18T13:51:51.767339+0000","last_fullsized":"2024-08-18T13:51:51.767339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072121939999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"54'77","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:17.643221+0000","last_change":"2024-08-18T13:52:17.643163+0000","last_active":"2024-08-18T13:52:17.643221+0000","last_peered":"2024-08-18T13:52:17.643221+0000","last_clean":"2024-08-18T13:52:17.643221+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:52:17.643221+0000","last_undegraded":"2024-08-18T13:52:17.643221+0000","last_fullsized":"2024-08-18T13:52:17.643221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'77","last_scrub_stamp":"2024-08-18T13:52:17.642974+0000","last_deep_scrub":"54'77","last_deep_scrub_stamp":"2024-08-18T13:52:17.642974+0000","last_clean_scrub_stamp":"2024-08-18T13:52:17.642974+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:17.078190+0000","scrub_duration":0.0036672470000000002,"objects_trimmed":0,"snaptrim_duration":0.0057680220000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"54'90","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777396+0000","last_change":"2024-08-18T13:43:55.958601+0000","last_active":"2024-08-18T13:51:51.777396+0000","last_peered":"2024-08-18T13:51:51.777396+0000","last_clean":"2024-08-18T13:51:51.777396+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:51:51.777396+0000","last_undegraded":"2024-08-18T13:51:51.777396+0000","last_fullsized":"2024-08-18T13:51:51.777396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049025989999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"54'98","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.780172+0000","last_change":"2024-08-18T13:43:55.959710+0000","last_active":"2024-08-18T13:51:51.780172+0000","last_peered":"2024-08-18T13:51:51.780172+0000","last_clean":"2024-08-18T13:51:51.780172+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:51:51.780172+0000","last_undegraded":"2024-08-18T13:51:51.780172+0000","last_fullsized":"2024-08-18T13:51:51.780172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029569129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"54'83","reported_seq":2147,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781572+0000","last_change":"2024-08-18T13:43:55.552920+0000","last_active":"2024-08-18T13:51:51.781572+0000","last_peered":"2024-08-18T13:51:51.781572+0000","last_clean":"2024-08-18T13:51:51.781572+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:51:51.781572+0000","last_undegraded":"2024-08-18T13:51:51.781572+0000","last_fullsized":"2024-08-18T13:51:51.781572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062586680000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"54'74","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.923219+0000","last_change":"2024-08-18T13:52:04.923145+0000","last_active":"2024-08-18T13:52:04.923219+0000","last_peered":"2024-08-18T13:52:04.923219+0000","last_clean":"2024-08-18T13:52:04.923219+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:52:04.923219+0000","last_undegraded":"2024-08-18T13:52:04.923219+0000","last_fullsized":"2024-08-18T13:52:04.923219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'74","last_scrub_stamp":"2024-08-18T13:52:04.922933+0000","last_deep_scrub":"54'74","last_deep_scrub_stamp":"2024-08-18T13:52:04.922933+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.922933+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:50.961793+0000","scrub_duration":0.0036959599999999999,"objects_trimmed":0,"snaptrim_duration":0.0064056340000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"54'105","reported_seq":2169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778715+0000","last_change":"2024-08-18T13:43:55.554037+0000","last_active":"2024-08-18T13:51:51.778715+0000","last_peered":"2024-08-18T13:51:51.778715+0000","last_clean":"2024-08-18T13:51:51.778715+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:51:51.778715+0000","last_undegraded":"2024-08-18T13:51:51.778715+0000","last_fullsized":"2024-08-18T13:51:51.778715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050911749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"54'93","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.777040+0000","last_change":"2024-08-18T13:43:55.862998+0000","last_active":"2024-08-18T13:51:51.777040+0000","last_peered":"2024-08-18T13:51:51.777040+0000","last_clean":"2024-08-18T13:51:51.777040+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:51:51.777040+0000","last_undegraded":"2024-08-18T13:51:51.777040+0000","last_fullsized":"2024-08-18T13:51:51.777040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0068654359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"54'102","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.781340+0000","last_change":"2024-08-18T13:43:55.550186+0000","last_active":"2024-08-18T13:51:51.781340+0000","last_peered":"2024-08-18T13:51:51.781340+0000","last_clean":"2024-08-18T13:51:51.781340+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:51:51.781340+0000","last_undegraded":"2024-08-18T13:51:51.781340+0000","last_fullsized":"2024-08-18T13:51:51.781340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055002549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"54'76","reported_seq":2141,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.775083+0000","last_change":"2024-08-18T13:43:55.859954+0000","last_active":"2024-08-18T13:51:51.775083+0000","last_peered":"2024-08-18T13:51:51.775083+0000","last_clean":"2024-08-18T13:51:51.775083+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:51:51.775083+0000","last_undegraded":"2024-08-18T13:51:51.775083+0000","last_fullsized":"2024-08-18T13:51:51.775083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069338170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.940841+0000","last_change":"2024-08-18T13:52:20.940751+0000","last_active":"2024-08-18T13:52:20.940841+0000","last_peered":"2024-08-18T13:52:20.940841+0000","last_clean":"2024-08-18T13:52:20.940841+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:52:20.940841+0000","last_undegraded":"2024-08-18T13:52:20.940841+0000","last_fullsized":"2024-08-18T13:52:20.940841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:20.940535+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:20.940535+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.940535+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:23.879899+0000","scrub_duration":0.003381294,"objects_trimmed":0,"snaptrim_duration":0.0044622389999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"54'76","reported_seq":2141,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.763906+0000","last_change":"2024-08-18T13:43:56.136824+0000","last_active":"2024-08-18T13:51:51.763906+0000","last_peered":"2024-08-18T13:51:51.763906+0000","last_clean":"2024-08-18T13:51:51.763906+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:51:51.763906+0000","last_undegraded":"2024-08-18T13:51:51.763906+0000","last_fullsized":"2024-08-18T13:51:51.763906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035965799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"54'86","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778898+0000","last_change":"2024-08-18T13:43:55.552484+0000","last_active":"2024-08-18T13:51:51.778898+0000","last_peered":"2024-08-18T13:51:51.778898+0000","last_clean":"2024-08-18T13:51:51.778898+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:51:51.778898+0000","last_undegraded":"2024-08-18T13:51:51.778898+0000","last_fullsized":"2024-08-18T13:51:51.778898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067690759999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":938443088,"num_objects":10906,"num_object_clones":0,"num_object_copies":43624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10906,"num_whiteouts":0,"num_read":21895,"num_read_kb":705026,"num_write":21484,"num_write_kb":1061752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2001649664,"data_stored":2001649664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5769,"ondisk_log_size":5769,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":184848248,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":586,"num_read_kb":685208,"num_write":6435,"num_write_kb":186300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":544529,"num_omap_keys":1040,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":369762304,"data_stored":369696496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6475,"ondisk_log_size":6475,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509600,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600108,"kb_used_data":294436,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93771732,"statfs":{"total":96636764160,"available":96022253568,"internally_reserved":0,"allocated":301502464,"data_stored":301200284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2999999999999998,"5min":1.026,"15min":0.878},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":25.744,"5min":25.744,"15min":25.744},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.3149999999999999,"5min":0.84999999999999998,"15min":0.82599999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.33100000000000002},"max":{"1min":8.1679999999999993,"5min":8.1679999999999993,"15min":8.1679999999999993},"last":1.0860000000000001}]},{"osd":1,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":0.90700000000000003,"15min":0.98599999999999999},"min":{"1min":0.5,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":10.151999999999999,"5min":10.151999999999999,"15min":39.203000000000003},"last":0.878},{"interface":"front","average":{"1min":2.323,"5min":1.111,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":25.852,"5min":25.852,"15min":25.852},"last":0.79700000000000004}]},{"osd":2,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":1.121,"15min":0.95099999999999996},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.39400000000000002},"max":{"1min":25.818999999999999,"5min":25.818999999999999,"15min":25.818999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.85199999999999998},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.35799999999999998},"max":{"1min":1.133,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.0840000000000001,"15min":1.135},"min":{"1min":0.64700000000000002,"5min":0.42699999999999999,"15min":0.377},"max":{"1min":17.300999999999998,"5min":17.300999999999998,"15min":42.905000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":2.4689999999999999,"5min":1.238,"15min":1.115},"min":{"1min":0.75600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":25.786999999999999,"5min":25.786999999999999,"15min":25.786999999999999},"last":1.056}]},{"osd":4,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.29699999999999999},"max":{"1min":8.3870000000000005,"5min":8.3870000000000005,"15min":8.3870000000000005},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.171,"15min":1.23},"min":{"1min":0.79800000000000004,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":17.056999999999999,"5min":17.056999999999999,"15min":52.673000000000002},"last":0.83199999999999996}]},{"osd":5,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":1.274,"15min":1.119},"min":{"1min":0.44,"5min":0.44,"15min":0.33700000000000002},"max":{"1min":25.881,"5min":25.881,"15min":25.881},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.1080000000000001,"15min":1.073},"min":{"1min":0.73899999999999999,"5min":0.54200000000000004,"15min":0.37},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.1659999999999999,"15min":1.1259999999999999},"min":{"1min":0.83799999999999997,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":8.0510000000000002,"5min":8.0510000000000002,"15min":8.0510000000000002},"last":1.026},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.1990000000000001,"15min":1.26},"min":{"1min":0.85499999999999998,"5min":0.49399999999999999,"15min":0.435},"max":{"1min":10.523,"5min":10.523,"15min":39.161000000000001},"last":1.0029999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509601,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610072,"kb_used_data":312720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93761768,"statfs":{"total":96636764160,"available":96012050432,"internally_reserved":0,"allocated":320225280,"data_stored":319926507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76500000000000001,"15min":0.88},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.165,"5min":1.21,"15min":4.532},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.248},"max":{"1min":1.258,"5min":1.3200000000000001,"15min":1.9650000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.92000000000000004},"min":{"1min":0.42399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2749999999999999,"5min":1.294,"15min":4.468},"last":1.179},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.91900000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.30499999999999999},"max":{"1min":1.1990000000000001,"5min":1.286,"15min":2.3620000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77700000000000002,"15min":0.83899999999999997},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.28000000000000003},"max":{"1min":1.085,"5min":1.177,"15min":1.413},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.97299999999999998},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1060000000000001,"5min":1.468,"15min":4.5},"last":0.96899999999999997}]},{"osd":3,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88700000000000001,"15min":0.94199999999999995},"min":{"1min":0.60899999999999999,"5min":0.41999999999999998,"15min":0.35399999999999998},"max":{"1min":1.1919999999999999,"5min":1.276,"15min":4.3250000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.91600000000000004},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.32900000000000001},"max":{"1min":1.0920000000000001,"5min":1.306,"15min":1.8069999999999999},"last":1.02}]},{"osd":4,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92700000000000005,"15min":1.0109999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.228,"5min":1.552,"15min":5.2549999999999999},"last":1.093},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.7230000000000001,"15min":4.6600000000000001},"last":0.99399999999999999}]},{"osd":5,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":1.0129999999999999},"min":{"1min":0.54900000000000004,"5min":0.45400000000000001,"15min":0.373},"max":{"1min":1.179,"5min":1.3160000000000001,"15min":4.4009999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.042,"15min":1.1499999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.272,"5min":1.6879999999999999,"15min":6.1260000000000003},"last":1.244}]},{"osd":7,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0669999999999999,"15min":1.111},"min":{"1min":0.67600000000000005,"5min":0.55500000000000005,"15min":0.40500000000000003},"max":{"1min":1.3009999999999999,"5min":1.4830000000000001,"15min":1.9299999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0609999999999999,"15min":1.1080000000000001},"min":{"1min":0.67500000000000004,"5min":0.46300000000000002,"15min":0.40200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":2.1150000000000002},"last":1.1319999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509599,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570388,"kb_used_data":273036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93801452,"statfs":{"total":96636764160,"available":96052686848,"internally_reserved":0,"allocated":279588864,"data_stored":279281600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.24299999999999999},"max":{"1min":1.1679999999999999,"5min":1.204,"15min":1.22},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":1.085,"5min":1.339,"15min":1.73},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82499999999999996,"15min":0.86799999999999999},"min":{"1min":0.54100000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.2909999999999999,"5min":1.415,"15min":3.6299999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.36699999999999999,"15min":0.27600000000000002},"max":{"1min":1.0589999999999999,"5min":1.28,"15min":1.5169999999999999},"last":0.85899999999999999}]},{"osd":2,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.85799999999999998},"min":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.5920000000000001},"last":0.91500000000000004}]},{"osd":3,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92100000000000004,"15min":0.91800000000000004},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.309},"max":{"1min":1.111,"5min":1.387,"15min":1.45},"last":1.135},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.641,"15min":1.641},"last":0.97999999999999998}]},{"osd":5,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.313},"max":{"1min":1.3640000000000001,"5min":1.575,"15min":3.7789999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.998,"15min":1.0149999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3129999999999999,"5min":1.754,"15min":1.754},"last":1.2030000000000001}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0580000000000001,"15min":1.079},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.504},"max":{"1min":1.6259999999999999,"5min":1.901,"15min":7.8490000000000002},"last":1.2390000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.016,"15min":1.0629999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2809999999999999,"5min":1.5229999999999999,"15min":6.3380000000000001},"last":1.2589999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40000000000000002},"max":{"1min":1.2470000000000001,"5min":1.7470000000000001,"15min":1.8400000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.151,"15min":1.1819999999999999},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.42399999999999999},"max":{"1min":1.6850000000000001,"5min":2.4569999999999999,"15min":3.7410000000000001},"last":1.2849999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542304,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683436,"kb_used_data":373604,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93688404,"statfs":{"total":96636764160,"available":95936925696,"internally_reserved":0,"allocated":382570496,"data_stored":382269906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.29499999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.89500000000000002,"5min":1.2050000000000001,"15min":1.2170000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67800000000000005,"15min":0.69299999999999995},"min":{"1min":0.44900000000000001,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":1.413,"15min":1.413},"last":0.68799999999999994}]},{"osd":1,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.68600000000000005},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86499999999999999,"5min":1.46,"15min":3.4820000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.751},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.8979999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.48599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1910000000000001,"5min":1.6419999999999999,"15min":2.415},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.87,"15min":0.86899999999999999},"min":{"1min":0.57799999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.2589999999999999,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.66500000000000004}]},{"osd":4,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.173,"5min":1.681,"15min":3.2490000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.052},"min":{"1min":0.66300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5189999999999999,"5min":2.4870000000000001,"15min":3.8799999999999999},"last":0.96799999999999997}]},{"osd":5,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.996,"15min":4.1059999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0089999999999999,"15min":1.05},"min":{"1min":0.63200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1719999999999999,"5min":1.627,"15min":3.2799999999999998},"last":1.091}]},{"osd":6,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.504,"5min":0.504,"15min":0.41999999999999998},"max":{"1min":1.3480000000000001,"5min":1.833,"15min":2.1080000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.054,"15min":1.0880000000000001},"min":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.573,"15min":2.7469999999999999},"last":1.153}]},{"osd":7,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4419999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.087,"15min":1.0960000000000001},"min":{"1min":0.71499999999999997,"5min":0.63200000000000001,"15min":0.51100000000000001},"max":{"1min":1.583,"5min":1.9039999999999999,"15min":2.1429999999999998},"last":1.0269999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542303,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530780,"kb_used_data":229268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93841060,"statfs":{"total":96636764160,"available":96093245440,"internally_reserved":0,"allocated":234770432,"data_stored":234477374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.186},"max":{"1min":1.0369999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":1.0960000000000001,"5min":3.27,"15min":3.27},"last":0.60799999999999998}]},{"osd":1,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.038,"5min":1.1040000000000001,"15min":1.696},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":3.2290000000000001,"15min":3.2290000000000001},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.78200000000000003},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.253},"max":{"1min":0.995,"5min":1.077,"15min":12.468999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.84399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35899999999999999,"15min":0.27500000000000002},"max":{"1min":1.077,"5min":1.3540000000000001,"15min":1.744},"last":0.74199999999999999}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2709999999999999,"5min":1.681,"15min":1.8480000000000001},"last":1.194},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.94799999999999995},"min":{"1min":0.46899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2609999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1759999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":1.113},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39000000000000001},"max":{"1min":1.149,"5min":1.498,"15min":1.627},"last":1.0429999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96899999999999997,"15min":0.98799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.42199999999999999},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.649},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.0309999999999999,"15min":1.0649999999999999},"min":{"1min":0.67000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":1.5820000000000001,"15min":1.905},"last":1.0780000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0549999999999999,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.60399999999999998,"15min":0.436},"max":{"1min":1.3140000000000001,"5min":1.5089999999999999,"15min":1.9490000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.0149999999999999},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.46400000000000002},"max":{"1min":1.343,"5min":1.4379999999999999,"15min":1.7669999999999999},"last":0.96799999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542303,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677976,"kb_used_data":376464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93693864,"statfs":{"total":96636764160,"available":95942516736,"internally_reserved":0,"allocated":385499136,"data_stored":385199863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.65200000000000002},"min":{"1min":0.28899999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.94899999999999995,"5min":1.3009999999999999,"15min":3.1160000000000001},"last":0.747},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.72099999999999997},"min":{"1min":0.30599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.107,"5min":1.327,"15min":3.718},"last":1.0940000000000001}]},{"osd":2,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0549999999999999,"5min":4.968,"15min":4.968},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33400000000000002,"15min":0.20899999999999999},"max":{"1min":1.327,"5min":1.3560000000000001,"15min":9.3309999999999995},"last":1.008}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4810000000000001,"15min":1.927},"last":1.141},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79400000000000004,"15min":0.84899999999999998},"min":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":1.46,"5min":1.46,"15min":6.8559999999999999},"last":0.98299999999999998}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.28000000000000003},"max":{"1min":1.488,"5min":5.1020000000000003,"15min":5.6529999999999996},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.036,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.63400000000000001,"5min":0.39700000000000002,"15min":0.28599999999999998},"max":{"1min":1.4410000000000001,"5min":1.554,"15min":1.5740000000000001},"last":1.2110000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3859999999999999,"5min":1.546,"15min":3.7570000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1,"5min":1.04,"15min":1.1040000000000001},"min":{"1min":0.47799999999999998,"5min":0.46400000000000002,"15min":0.45700000000000002},"max":{"1min":1.591,"5min":5.1799999999999997,"15min":19.027999999999999},"last":1.0660000000000001}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.40699999999999997},"max":{"1min":1.5529999999999999,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.113,"5min":1.0880000000000001,"15min":1.131},"min":{"1min":0.54900000000000004,"5min":0.49099999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.861},"last":1.2689999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.069,"15min":1.119},"min":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58799999999999997},"max":{"1min":1.383,"5min":1.526,"15min":3.8100000000000001},"last":1.452},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40000000000000002},"max":{"1min":1.431,"5min":5.3099999999999996,"15min":5.3099999999999996},"last":1.129}]}]},{"osd":0,"up_from":14,"seq":60129542303,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549780,"kb_used_data":248268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93822060,"statfs":{"total":96636764160,"available":96073789440,"internally_reserved":0,"allocated":254226432,"data_stored":253919977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.65400000000000003},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":5.3159999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.33500000000000002,"5min":0.254,"15min":0.19500000000000001},"max":{"1min":0.93600000000000005,"5min":1.2370000000000001,"15min":3.4809999999999999},"last":0.96499999999999997}]},{"osd":2,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.47099999999999997,"5min":0.32600000000000001,"15min":0.26300000000000001},"max":{"1min":0.99199999999999999,"5min":1.222,"15min":3.3029999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.28},"last":0.76700000000000002}]},{"osd":3,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.288,"5min":1.288,"15min":5.399},"last":0.751},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86799999999999999,"15min":0.93100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.33900000000000002},"max":{"1min":1.1799999999999999,"5min":1.365,"15min":5.0359999999999996},"last":1.048}]},{"osd":4,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.37},"max":{"1min":1.3899999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":8.7799999999999994},"last":1.264}]},{"osd":5,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96099999999999997,"15min":0.97499999999999998},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.40300000000000002},"max":{"1min":1.51,"5min":1.52,"15min":1.8879999999999999},"last":1.347}]},{"osd":6,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.77800000000000002,"5min":0.46500000000000002,"15min":0.374},"max":{"1min":1.423,"5min":1.599,"15min":3.7200000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.042,"15min":1.085},"min":{"1min":0.68600000000000005,"5min":0.58099999999999996,"15min":0.52600000000000002},"max":{"1min":1.55,"5min":1.659,"15min":3.2789999999999999},"last":1.103}]},{"osd":7,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.143,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.60599999999999998,"15min":0.40300000000000002},"max":{"1min":1.514,"5min":1.704,"15min":36.295000000000002},"last":1.22},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.171,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.55300000000000005,"15min":0.42999999999999999},"max":{"1min":1.5740000000000001,"5min":1.706,"15min":3.448},"last":1.365}]}]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515540,"kb_used_data":214028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93856300,"statfs":{"total":96636764160,"available":96108851200,"internally_reserved":0,"allocated":219164672,"data_stored":218869425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.81100000000000005},"min":{"1min":0.48199999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.135,"5min":1.2789999999999999,"15min":1.901},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.81000000000000005,"15min":0.85599999999999998},"min":{"1min":0.63900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":4.0810000000000004},"last":0.98799999999999999}]},{"osd":1,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.82099999999999995},"min":{"1min":0.51900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.101,"5min":1.429,"15min":2.4809999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.89800000000000002,"15min":0.91600000000000004},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1799999999999999,"5min":1.456,"15min":1.7749999999999999},"last":1.0229999999999999}]},{"osd":2,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.63200000000000001,"5min":0.46999999999999997,"15min":0.41799999999999998},"max":{"1min":1.1419999999999999,"5min":1.524,"15min":1.774},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.874,"15min":0.93899999999999995},"min":{"1min":0.61499999999999999,"5min":0.44600000000000001,"15min":0.32500000000000001},"max":{"1min":1.1299999999999999,"5min":1.3160000000000001,"15min":4.1559999999999997},"last":0.84699999999999998}]},{"osd":3,"last update":"Sun Aug 18 13:51:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.94399999999999995,"15min":0.97999999999999998},"min":{"1min":0.59799999999999998,"5min":0.54600000000000004,"15min":0.36199999999999999},"max":{"1min":1.3300000000000001,"5min":1.4410000000000001,"15min":4.5510000000000002},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.92700000000000005,"15min":0.98499999999999999},"min":{"1min":0.72199999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.423,"5min":1.4470000000000001,"15min":3.976},"last":1.0600000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.7909999999999999},"last":1.121},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0660000000000001,"15min":1.1120000000000001},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.6839999999999999,"15min":2.532},"last":1.103}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.03,"15min":1.085},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3109999999999999,"5min":1.532,"15min":4.2050000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0569999999999999,"15min":1.113},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":2.911},"last":0.77000000000000002}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.089,"15min":1.165},"min":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2989999999999999,"5min":1.659,"15min":2.569},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.1359999999999999,"15min":1.361},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40400000000000003},"max":{"1min":1.407,"5min":1.605,"15min":58.290999999999997},"last":1.2270000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41955328,"data_stored":41943666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50331808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21549056,"data_stored":21547239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34140160,"data_stored":34130811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":54263808,"data_stored":54247785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41947136,"data_stored":41943130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50332308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":75227136,"data_stored":75219749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211058688,"data_stored":211058688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334540800,"data_stored":334540800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212602880,"data_stored":212602880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347811840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":224706560,"data_stored":224706560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176599040,"data_stored":176599040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":269266944,"data_stored":269266944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":225062912,"data_stored":225062912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:52:24.394 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.395 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.395 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-08-18T13:39:33.349883+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=33, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.395 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.395 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.396 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.396 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.396 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.396 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.396 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.397 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.397 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.397 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.397 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.397 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.398 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.398 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.398 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.398 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.398 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.398 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.399 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.399 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.399 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.399 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.399 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.400 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.400 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.400 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.400 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.400 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.400 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.401 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.401 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.401 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.402 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.402 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.402 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.403 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.403 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.403 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.403 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.404 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.404 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.404 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.404 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.404 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.405 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.405 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.405 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.405 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.406 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.406 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.406 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.406 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.406 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.407 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.407 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.407 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.407 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.408 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.408 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.408 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.408 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.408 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:24.408 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-18T13:52:44.410 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:52:44.520 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.513+0000 7fc9d3fff700 1 Processor -- start 2024-08-18T13:52:44.521 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.513+0000 7fc9d3fff700 1 -- start start 2024-08-18T13:52:44.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40aebc0 0x7fc9d40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:44.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d4155330 con 0x7fc9d40c6880 2024-08-18T13:52:44.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d41555d0 con 0x7fc9d40aef90 2024-08-18T13:52:44.523 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4155140 con 0x7fc9d40aebc0 2024-08-18T13:52:44.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d37fe700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fc9d40c6880 0x7fc9d40c6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:52066/0 (socket says 172.21.15.118:52066) 2024-08-18T13:52:44.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d37fe700 1 -- 172.21.15.118:0/182512576 learned_addr learned my addr 172.21.15.118:0/182512576 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:52:44.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d2ffd700 1 --2- 172.21.15.118:0/182512576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40aebc0 0x7fc9d40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:44.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148385519 0 0) 0x7fc9d4155330 con 0x7fc9d40c6880 2024-08-18T13:52:44.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9bc003570 con 0x7fc9d40c6880 2024-08-18T13:52:44.524 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166108687 0 0) 0x7fc9d41555d0 con 0x7fc9d40aef90 2024-08-18T13:52:44.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d4155330 con 0x7fc9d40aef90 2024-08-18T13:52:44.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3950628151 0 0) 0x7fc9bc003570 con 0x7fc9d40c6880 2024-08-18T13:52:44.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d2ffd700 1 -- 172.21.15.118:0/182512576 >> v1:172.21.15.178:6789/0 conn(0x7fc9d40aef90 legacy=0x7fc9d40abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:44.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d2ffd700 1 -- 172.21.15.118:0/182512576 >> v1:172.21.15.118:6789/0 conn(0x7fc9d40c6880 legacy=0x7fc9d40c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:44.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d2ffd700 1 -- 172.21.15.118:0/182512576 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4155870 con 0x7fc9d40aebc0 2024-08-18T13:52:44.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d2ffd700 1 --2- 172.21.15.118:0/182512576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40aebc0 0x7fc9d40b1640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009e90 tx=0x7fc9cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dff056ebb7f75a6a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:44.525 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00e070 con 0x7fc9d40aebc0 2024-08-18T13:52:44.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc00b870 con 0x7fc9d40aebc0 2024-08-18T13:52:44.526 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d1ffb700 1 -- 172.21.15.118:0/182512576 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4001470 con 0x7fc9d40aebc0 2024-08-18T13:52:44.526 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:52:44.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- 172.21.15.118:0/182512576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40aebc0 msgr2=0x7fc9d40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:44.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/182512576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40aebc0 0x7fc9d40b1640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009e90 tx=0x7fc9cc009780 comp rx=0 tx=0).stop 2024-08-18T13:52:44.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- 172.21.15.118:0/182512576 shutdown_connections 2024-08-18T13:52:44.532 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/182512576 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40aebc0 0x7fc9d40b1640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:44.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- 172.21.15.118:0/182512576 >> 172.21.15.118:0/182512576 conn(0x7fc9d40cbba0 msgr2=0x7fc9d40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:44.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- 172.21.15.118:0/182512576 shutdown_connections 2024-08-18T13:52:44.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.517+0000 7fc9d3fff700 1 -- 172.21.15.118:0/182512576 wait complete. 2024-08-18T13:52:44.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 Processor -- start 2024-08-18T13:52:44.533 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 -- start start 2024-08-18T13:52:44.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9d40aebc0 0x7fc9d414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:44.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9d40aef90 0x7fc9d4068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:44.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40c6880 0x7fc9d414d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:44.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40694e0 con 0x7fc9d40aef90 2024-08-18T13:52:44.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40683a0 con 0x7fc9d40aebc0 2024-08-18T13:52:44.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d3fff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4166150 con 0x7fc9d40c6880 2024-08-18T13:52:44.534 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d27fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9d40aef90 0x7fc9d4068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:44.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d27fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9d40aef90 0x7fc9d4068ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:54210/0 (socket says 172.21.15.118:54210) 2024-08-18T13:52:44.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d27fc700 1 -- 172.21.15.118:0/509108817 learned_addr learned my addr 172.21.15.118:0/509108817 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:52:44.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d2ffd700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9d40aebc0 0x7fc9d414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:44.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.521+0000 7fc9d37fe700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40c6880 0x7fc9d414d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:44.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d27fc700 1 -- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40c6880 msgr2=0x7fc9d414d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:44.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d27fc700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40c6880 0x7fc9d414d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:44.535 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d27fc700 1 -- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9d40aebc0 msgr2=0x7fc9d414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:44.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d27fc700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9d40aebc0 0x7fc9d414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:44.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d27fc700 1 -- 172.21.15.118:0/509108817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc009a20 con 0x7fc9d40aef90 2024-08-18T13:52:44.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d27fc700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9d40aef90 0x7fc9d4068ad0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c790 tx=0x7fc9c8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:44.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9baffd700 1 -- 172.21.15.118:0/509108817 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8010510 con 0x7fc9d40aef90 2024-08-18T13:52:44.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9baffd700 1 -- 172.21.15.118:0/509108817 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8010670 con 0x7fc9d40aef90 2024-08-18T13:52:44.536 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d4068640 con 0x7fc9d40aef90 2024-08-18T13:52:44.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9baffd700 1 -- 172.21.15.118:0/509108817 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8010a10 con 0x7fc9d40aef90 2024-08-18T13:52:44.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4165a10 con 0x7fc9d40aef90 2024-08-18T13:52:44.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d40687d0 con 0x7fc9d40aef90 2024-08-18T13:52:44.537 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9baffd700 1 -- 172.21.15.118:0/509108817 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc9c801b3b0 con 0x7fc9d40aef90 2024-08-18T13:52:44.538 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:52:44.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.525+0000 7fc9baffd700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc9c0060890 0x7fc9c0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:52:44.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.529+0000 7fc9baffd700 1 -- 172.21.15.118:0/509108817 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fc9c807ca40 con 0x7fc9d40aef90 2024-08-18T13:52:44.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.529+0000 7fc9d2ffd700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc9c0060890 0x7fc9c0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:52:44.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.529+0000 7fc9d2ffd700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc9c0060890 0x7fc9c0062d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0097c0 tx=0x7fc9cc017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:52:44.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.533+0000 7fc9baffd700 1 -- 172.21.15.118:0/509108817 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc9c804c830 con 0x7fc9d40aef90 2024-08-18T13:52:44.700 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.693+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9d40713a0 con 0x7fc9c0060890 2024-08-18T13:52:44.718 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.713+0000 7fc9baffd700 1 -- 172.21.15.118:0/509108817 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+374556 (secure 0 0 0) 0x7fc9d40713a0 con 0x7fc9c0060890 2024-08-18T13:52:44.723 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:52:44.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc9c0060890 msgr2=0x7fc9c0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:44.726 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc9c0060890 0x7fc9c0062d40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0097c0 tx=0x7fc9cc017000 comp rx=0 tx=0).stop 2024-08-18T13:52:44.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9d40aef90 msgr2=0x7fc9d4068ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:52:44.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9d40aef90 0x7fc9d4068ad0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c790 tx=0x7fc9c8007e10 comp rx=0 tx=0).stop 2024-08-18T13:52:44.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 shutdown_connections 2024-08-18T13:52:44.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc9c0060890 0x7fc9c0062d40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:44.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9d40c6880 0x7fc9d414d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:44.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc9d40aef90 0x7fc9d4068ad0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:44.727 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 --2- 172.21.15.118:0/509108817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9d40aebc0 0x7fc9d414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:52:44.728 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 >> 172.21.15.118:0/509108817 conn(0x7fc9d40cbba0 msgr2=0x7fc9d40ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:52:44.728 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 shutdown_connections 2024-08-18T13:52:44.728 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:52:44.717+0000 7fc9d3fff700 1 -- 172.21.15.118:0/509108817 wait complete. 2024-08-18T13:52:44.728 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:52:44.738 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-08-18T13:52:43.506546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1123881704,"num_objects":11047,"num_object_clones":0,"num_object_copies":43906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11047,"num_whiteouts":0,"num_read":22597,"num_read_kb":1390334,"num_write":28078,"num_write_kb":1250468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576095,"num_omap_keys":1100,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33897,"ondisk_log_size":33897,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4738080,"kb_used_data":2321824,"kb_used_omap":13,"kb_used_meta":2416178,"kb_avail":750236640,"statfs":{"total":773094113280,"available":768242319360,"internally_reserved":0,"allocated":2377547776,"data_stored":2375144936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2474166976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005124"},"pg_stats":[{"pgid":"2.1f","version":"54'156","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.111340+0000","last_change":"2024-08-18T13:52:07.111260+0000","last_active":"2024-08-18T13:52:07.111340+0000","last_peered":"2024-08-18T13:52:07.111340+0000","last_clean":"2024-08-18T13:52:07.111340+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:52:07.111340+0000","last_undegraded":"2024-08-18T13:52:07.111340+0000","last_fullsized":"2024-08-18T13:52:07.111340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'156","last_scrub_stamp":"2024-08-18T13:52:07.111121+0000","last_deep_scrub":"54'156","last_deep_scrub_stamp":"2024-08-18T13:52:07.111121+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.111121+0000","objects_scrubbed":5,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:38.124367+0000","scrub_duration":0.025498494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389178,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":28696,"num_write":150,"num_write_kb":8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11702,"num_omap_keys":22,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.258859+0000","last_change":"2024-08-18T13:52:27.258755+0000","last_active":"2024-08-18T13:52:27.258859+0000","last_peered":"2024-08-18T13:52:27.258859+0000","last_clean":"2024-08-18T13:52:27.258859+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:52:27.258859+0000","last_undegraded":"2024-08-18T13:52:27.258859+0000","last_fullsized":"2024-08-18T13:52:27.258859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:27.258561+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:27.258561+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.258561+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:05.192013+0000","scrub_duration":0.0038873929999999998,"objects_trimmed":0,"snaptrim_duration":0.0064850389999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"54'676","reported_seq":3752,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.851363+0000","last_change":"2024-08-18T13:43:55.552107+0000","last_active":"2024-08-18T13:51:49.851363+0000","last_peered":"2024-08-18T13:51:49.851363+0000","last_clean":"2024-08-18T13:51:49.851363+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:51:49.851363+0000","last_undegraded":"2024-08-18T13:51:49.851363+0000","last_fullsized":"2024-08-18T13:51:49.851363+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067303959999999996,"stat_sum":{"num_bytes":37566158,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":1013,"num_read_kb":36248,"num_write":673,"num_write_kb":41139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"54'79","reported_seq":138,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.012943+0000","last_change":"2024-08-18T13:52:14.012891+0000","last_active":"2024-08-18T13:52:14.012943+0000","last_peered":"2024-08-18T13:52:14.012943+0000","last_clean":"2024-08-18T13:52:14.012943+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:52:14.012943+0000","last_undegraded":"2024-08-18T13:52:14.012943+0000","last_fullsized":"2024-08-18T13:52:14.012943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'79","last_scrub_stamp":"2024-08-18T13:52:14.012738+0000","last_deep_scrub":"54'79","last_deep_scrub_stamp":"2024-08-18T13:52:14.012738+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.012738+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:27:28.341973+0000","scrub_duration":0.015505003999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":75,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"54'87","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:01.076425+0000","last_change":"2024-08-18T13:52:01.076369+0000","last_active":"2024-08-18T13:52:01.076425+0000","last_peered":"2024-08-18T13:52:01.076425+0000","last_clean":"2024-08-18T13:52:01.076425+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:52:01.076425+0000","last_undegraded":"2024-08-18T13:52:01.076425+0000","last_fullsized":"2024-08-18T13:52:01.076425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_clean_scrub_stamp":"2024-08-18T13:52:01.076152+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:46:50.073762+0000","scrub_duration":0.0039656070000000003,"objects_trimmed":0,"snaptrim_duration":0.0075430439999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"54'705","reported_seq":3584,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.827339+0000","last_change":"2024-08-18T13:52:18.827164+0000","last_active":"2024-08-18T13:52:18.827339+0000","last_peered":"2024-08-18T13:52:18.827339+0000","last_clean":"2024-08-18T13:52:18.827339+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:52:18.827339+0000","last_undegraded":"2024-08-18T13:52:18.827339+0000","last_fullsized":"2024-08-18T13:52:18.827339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'705","last_scrub_stamp":"2024-08-18T13:52:18.826906+0000","last_deep_scrub":"54'705","last_deep_scrub_stamp":"2024-08-18T13:52:18.826906+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.826906+0000","objects_scrubbed":370,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:26:33.481234+0000","scrub_duration":0.157720147,"objects_trimmed":0,"snaptrim_duration":0.0075371290000000001,"stat_sum":{"num_bytes":29851183,"num_objects":370,"num_object_clones":0,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":792,"num_read_kb":20881,"num_write":708,"num_write_kb":37566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"54'202","reported_seq":278,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.924150+0000","last_change":"2024-08-18T13:52:22.924101+0000","last_active":"2024-08-18T13:52:22.924150+0000","last_peered":"2024-08-18T13:52:22.924150+0000","last_clean":"2024-08-18T13:52:22.924150+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:52:22.924150+0000","last_undegraded":"2024-08-18T13:52:22.924150+0000","last_fullsized":"2024-08-18T13:52:22.924150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'202","last_scrub_stamp":"2024-08-18T13:52:22.923937+0000","last_deep_scrub":"54'202","last_deep_scrub_stamp":"2024-08-18T13:52:22.923937+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.923937+0000","objects_scrubbed":8,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:49.870790+0000","scrub_duration":0.033031854999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":69650,"num_write":192,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31566,"num_omap_keys":60,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.557506+0000","last_change":"2024-08-18T13:52:27.557449+0000","last_active":"2024-08-18T13:52:27.557506+0000","last_peered":"2024-08-18T13:52:27.557506+0000","last_clean":"2024-08-18T13:52:27.557506+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:52:27.557506+0000","last_undegraded":"2024-08-18T13:52:27.557506+0000","last_fullsized":"2024-08-18T13:52:27.557506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:27.557280+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:27.557280+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.557280+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:04.966509+0000","scrub_duration":0.003068201,"objects_trimmed":0,"snaptrim_duration":0.0046085359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"54'680","reported_seq":3481,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.517007+0000","last_change":"2024-08-18T13:43:55.557895+0000","last_active":"2024-08-18T13:51:49.517007+0000","last_peered":"2024-08-18T13:51:49.517007+0000","last_clean":"2024-08-18T13:51:49.517007+0000","last_became_active":"2024-08-18T13:40:16.608743+0000","last_became_peered":"2024-08-18T13:40:16.608743+0000","last_unstale":"2024-08-18T13:51:49.517007+0000","last_undegraded":"2024-08-18T13:51:49.517007+0000","last_fullsized":"2024-08-18T13:51:49.517007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080275980000000004,"stat_sum":{"num_bytes":28774140,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":727,"num_read_kb":13856,"num_write":680,"num_write_kb":28426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"54'27","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.933094+0000","last_change":"2024-08-18T13:52:15.933048+0000","last_active":"2024-08-18T13:52:15.933094+0000","last_peered":"2024-08-18T13:52:15.933094+0000","last_clean":"2024-08-18T13:52:15.933094+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:52:15.933094+0000","last_undegraded":"2024-08-18T13:52:15.933094+0000","last_fullsized":"2024-08-18T13:52:15.933094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'27","last_scrub_stamp":"2024-08-18T13:52:15.932932+0000","last_deep_scrub":"54'27","last_deep_scrub_stamp":"2024-08-18T13:52:15.932932+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.932932+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:50:28.024428+0000","scrub_duration":0.016823828999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":25,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12356,"num_omap_keys":25,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"54'87","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.530882+0000","last_change":"2024-08-18T13:52:31.530815+0000","last_active":"2024-08-18T13:52:31.530882+0000","last_peered":"2024-08-18T13:52:31.530882+0000","last_clean":"2024-08-18T13:52:31.530882+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:52:31.530882+0000","last_undegraded":"2024-08-18T13:52:31.530882+0000","last_fullsized":"2024-08-18T13:52:31.530882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:31.530605+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:31.530605+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.530605+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:47:04.712973+0000","scrub_duration":0.0031435159999999998,"objects_trimmed":0,"snaptrim_duration":0.0065299970000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"54'674","reported_seq":2926,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.822593+0000","last_change":"2024-08-18T13:52:29.822494+0000","last_active":"2024-08-18T13:52:29.822593+0000","last_peered":"2024-08-18T13:52:29.822593+0000","last_clean":"2024-08-18T13:52:29.822593+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:52:29.822593+0000","last_undegraded":"2024-08-18T13:52:29.822593+0000","last_fullsized":"2024-08-18T13:52:29.822593+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'674","last_scrub_stamp":"2024-08-18T13:52:29.822290+0000","last_deep_scrub":"54'674","last_deep_scrub_stamp":"2024-08-18T13:52:29.822290+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.822290+0000","objects_scrubbed":345,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:23:06.162523+0000","scrub_duration":0.13583155499999999,"objects_trimmed":0,"snaptrim_duration":0.0078514629999999995,"stat_sum":{"num_bytes":11851705,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":166,"num_read_kb":1355,"num_write":673,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:24.298191+0000","last_change":"2024-08-18T13:52:24.298114+0000","last_active":"2024-08-18T13:52:24.298191+0000","last_peered":"2024-08-18T13:52:24.298191+0000","last_clean":"2024-08-18T13:52:24.298191+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:52:24.298191+0000","last_undegraded":"2024-08-18T13:52:24.298191+0000","last_fullsized":"2024-08-18T13:52:24.298191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:24.297918+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:24.297918+0000","last_clean_scrub_stamp":"2024-08-18T13:52:24.297918+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:46:26.475472+0000","scrub_duration":0.0035255989999999999,"objects_trimmed":0,"snaptrim_duration":0.0033699279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"54'252","reported_seq":322,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.940516+0000","last_change":"2024-08-18T13:52:21.940455+0000","last_active":"2024-08-18T13:52:21.940516+0000","last_peered":"2024-08-18T13:52:21.940516+0000","last_clean":"2024-08-18T13:52:21.940516+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:52:21.940516+0000","last_undegraded":"2024-08-18T13:52:21.940516+0000","last_fullsized":"2024-08-18T13:52:21.940516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'252","last_scrub_stamp":"2024-08-18T13:52:21.940309+0000","last_deep_scrub":"54'252","last_deep_scrub_stamp":"2024-08-18T13:52:21.940309+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.940309+0000","objects_scrubbed":9,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:44:34.338028+0000","scrub_duration":0.033709736999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12304619,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":23,"num_read_kb":45068,"num_write":239,"num_write_kb":12181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43761,"num_omap_keys":83,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"54'638","reported_seq":3282,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.659037+0000","last_change":"2024-08-18T13:43:55.558893+0000","last_active":"2024-08-18T13:51:49.659037+0000","last_peered":"2024-08-18T13:51:49.659037+0000","last_clean":"2024-08-18T13:51:49.659037+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:51:49.659037+0000","last_undegraded":"2024-08-18T13:51:49.659037+0000","last_fullsized":"2024-08-18T13:51:49.659037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059019540000000001,"stat_sum":{"num_bytes":16782769,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":581,"num_read_kb":14450,"num_write":639,"num_write_kb":24822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"54'87","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.890088+0000","last_change":"2024-08-18T13:52:08.890024+0000","last_active":"2024-08-18T13:52:08.890088+0000","last_peered":"2024-08-18T13:52:08.890088+0000","last_clean":"2024-08-18T13:52:08.890088+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:52:08.890088+0000","last_undegraded":"2024-08-18T13:52:08.890088+0000","last_fullsized":"2024-08-18T13:52:08.890088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:08.889811+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:08.889811+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.889811+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:42:55.362427+0000","scrub_duration":0.0034394989999999999,"objects_trimmed":0,"snaptrim_duration":0.0074252320000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"54'32","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:11.142035+0000","last_change":"2024-08-18T13:52:11.141993+0000","last_active":"2024-08-18T13:52:11.142035+0000","last_peered":"2024-08-18T13:52:11.142035+0000","last_clean":"2024-08-18T13:52:11.142035+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:52:11.142035+0000","last_undegraded":"2024-08-18T13:52:11.142035+0000","last_fullsized":"2024-08-18T13:52:11.142035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'32","last_scrub_stamp":"2024-08-18T13:52:11.141864+0000","last_deep_scrub":"54'32","last_deep_scrub_stamp":"2024-08-18T13:52:11.141864+0000","last_clean_scrub_stamp":"2024-08-18T13:52:11.141864+0000","objects_scrubbed":4,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:40.879760+0000","scrub_duration":0.016960533999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":28,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28048,"num_omap_keys":55,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"54'709","reported_seq":3823,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:35.849372+0000","last_change":"2024-08-18T13:52:35.849308+0000","last_active":"2024-08-18T13:52:35.849372+0000","last_peered":"2024-08-18T13:52:35.849372+0000","last_clean":"2024-08-18T13:52:35.849372+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:52:35.849372+0000","last_undegraded":"2024-08-18T13:52:35.849372+0000","last_fullsized":"2024-08-18T13:52:35.849372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'709","last_scrub_stamp":"2024-08-18T13:52:35.849145+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2024-08-18T13:52:35.849145+0000","last_clean_scrub_stamp":"2024-08-18T13:52:35.849145+0000","objects_scrubbed":347,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:20:01.973785+0000","scrub_duration":0.151375125,"objects_trimmed":0,"snaptrim_duration":0.0057029610000000003,"stat_sum":{"num_bytes":33413248,"num_objects":347,"num_object_clones":0,"num_object_copies":1388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1028,"num_read_kb":27410,"num_write":707,"num_write_kb":41185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:28.273167+0000","last_change":"2024-08-18T13:52:28.273103+0000","last_active":"2024-08-18T13:52:28.273167+0000","last_peered":"2024-08-18T13:52:28.273167+0000","last_clean":"2024-08-18T13:52:28.273167+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:52:28.273167+0000","last_undegraded":"2024-08-18T13:52:28.273167+0000","last_fullsized":"2024-08-18T13:52:28.273167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:28.272879+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:28.272879+0000","last_clean_scrub_stamp":"2024-08-18T13:52:28.272879+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:03.396288+0000","scrub_duration":0.0035927960000000001,"objects_trimmed":0,"snaptrim_duration":0.0069243719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"54'14","reported_seq":69,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:06.184482+0000","last_change":"2024-08-18T13:52:06.184363+0000","last_active":"2024-08-18T13:52:06.184482+0000","last_peered":"2024-08-18T13:52:06.184482+0000","last_clean":"2024-08-18T13:52:06.184482+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:52:06.184482+0000","last_undegraded":"2024-08-18T13:52:06.184482+0000","last_fullsized":"2024-08-18T13:52:06.184482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'14","last_scrub_stamp":"2024-08-18T13:52:06.184119+0000","last_deep_scrub":"54'14","last_deep_scrub_stamp":"2024-08-18T13:52:06.184119+0000","last_clean_scrub_stamp":"2024-08-18T13:52:06.184119+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:52.279744+0000","scrub_duration":0.013536823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":11,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'683","reported_seq":3571,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.731009+0000","last_change":"2024-08-18T13:43:56.142589+0000","last_active":"2024-08-18T13:51:49.731009+0000","last_peered":"2024-08-18T13:51:49.731009+0000","last_clean":"2024-08-18T13:51:49.731009+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:51:49.731009+0000","last_undegraded":"2024-08-18T13:51:49.731009+0000","last_fullsized":"2024-08-18T13:51:49.731009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079287170000000001,"stat_sum":{"num_bytes":20263748,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":825,"num_read_kb":25932,"num_write":679,"num_write_kb":28331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"54'92","reported_seq":2167,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.649198+0000","last_change":"2024-08-18T13:52:10.649127+0000","last_active":"2024-08-18T13:52:10.649198+0000","last_peered":"2024-08-18T13:52:10.649198+0000","last_clean":"2024-08-18T13:52:10.649198+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:52:10.649198+0000","last_undegraded":"2024-08-18T13:52:10.649198+0000","last_fullsized":"2024-08-18T13:52:10.649198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:10.648930+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:10.648930+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.648930+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:46:12.826484+0000","scrub_duration":0.0036575269999999998,"objects_trimmed":0,"snaptrim_duration":0.0068791520000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"54'110","reported_seq":168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.973413+0000","last_change":"2024-08-18T13:52:18.973337+0000","last_active":"2024-08-18T13:52:18.973413+0000","last_peered":"2024-08-18T13:52:18.973413+0000","last_clean":"2024-08-18T13:52:18.973413+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:52:18.973413+0000","last_undegraded":"2024-08-18T13:52:18.973413+0000","last_fullsized":"2024-08-18T13:52:18.973413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'110","last_scrub_stamp":"2024-08-18T13:52:18.973134+0000","last_deep_scrub":"54'110","last_deep_scrub_stamp":"2024-08-18T13:52:18.973134+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.973134+0000","objects_scrubbed":4,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:40.390138+0000","scrub_duration":0.045608213000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":108,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12312,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"54'617","reported_seq":3476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:41.711739+0000","last_change":"2024-08-18T13:52:41.711684+0000","last_active":"2024-08-18T13:52:41.711739+0000","last_peered":"2024-08-18T13:52:41.711739+0000","last_clean":"2024-08-18T13:52:41.711739+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:52:41.711739+0000","last_undegraded":"2024-08-18T13:52:41.711739+0000","last_fullsized":"2024-08-18T13:52:41.711739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'617","last_scrub_stamp":"2024-08-18T13:52:41.711559+0000","last_deep_scrub":"54'617","last_deep_scrub_stamp":"2024-08-18T13:52:41.711559+0000","last_clean_scrub_stamp":"2024-08-18T13:52:41.711559+0000","objects_scrubbed":338,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:19:56.474223+0000","scrub_duration":0.150787001,"objects_trimmed":0,"snaptrim_duration":0.004078028,"stat_sum":{"num_bytes":24272349,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":773,"num_read_kb":25488,"num_write":617,"num_write_kb":28093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'93","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.019178+0000","last_change":"2024-08-18T13:52:09.019114+0000","last_active":"2024-08-18T13:52:09.019178+0000","last_peered":"2024-08-18T13:52:09.019178+0000","last_clean":"2024-08-18T13:52:09.019178+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:52:09.019178+0000","last_undegraded":"2024-08-18T13:52:09.019178+0000","last_fullsized":"2024-08-18T13:52:09.019178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'93","last_scrub_stamp":"2024-08-18T13:52:09.018858+0000","last_deep_scrub":"54'93","last_deep_scrub_stamp":"2024-08-18T13:52:09.018858+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.018858+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:55.114483+0000","scrub_duration":0.0031236559999999998,"objects_trimmed":0,"snaptrim_duration":0.0060661889999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"54'114","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.265449+0000","last_change":"2024-08-18T13:52:05.265412+0000","last_active":"2024-08-18T13:52:05.265449+0000","last_peered":"2024-08-18T13:52:05.265449+0000","last_clean":"2024-08-18T13:52:05.265449+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:52:05.265449+0000","last_undegraded":"2024-08-18T13:52:05.265449+0000","last_fullsized":"2024-08-18T13:52:05.265449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'114","last_scrub_stamp":"2024-08-18T13:52:05.265313+0000","last_deep_scrub":"54'114","last_deep_scrub_stamp":"2024-08-18T13:52:05.265313+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.265313+0000","objects_scrubbed":4,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:57:48.580753+0000","scrub_duration":0.015757296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":111,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16148,"num_omap_keys":31,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"54'720","reported_seq":2994,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.759072+0000","last_change":"2024-08-18T13:52:21.758999+0000","last_active":"2024-08-18T13:52:21.759072+0000","last_peered":"2024-08-18T13:52:21.759072+0000","last_clean":"2024-08-18T13:52:21.759072+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:52:21.759072+0000","last_undegraded":"2024-08-18T13:52:21.759072+0000","last_fullsized":"2024-08-18T13:52:21.759072+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'720","last_scrub_stamp":"2024-08-18T13:52:21.758840+0000","last_deep_scrub":"54'720","last_deep_scrub_stamp":"2024-08-18T13:52:21.758840+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.758840+0000","objects_scrubbed":368,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:36.545679+0000","scrub_duration":0.120970787,"objects_trimmed":0,"snaptrim_duration":0.0079796720000000002,"stat_sum":{"num_bytes":17871290,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":187,"num_read_kb":7768,"num_write":717,"num_write_kb":21887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:26.213190+0000","last_change":"2024-08-18T13:52:26.213112+0000","last_active":"2024-08-18T13:52:26.213190+0000","last_peered":"2024-08-18T13:52:26.213190+0000","last_clean":"2024-08-18T13:52:26.213190+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:52:26.213190+0000","last_undegraded":"2024-08-18T13:52:26.213190+0000","last_fullsized":"2024-08-18T13:52:26.213190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:26.212910+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:26.212910+0000","last_clean_scrub_stamp":"2024-08-18T13:52:26.212910+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:34.964479+0000","scrub_duration":0.0037999150000000001,"objects_trimmed":0,"snaptrim_duration":0.0054340660000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"54'67","reported_seq":128,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.141636+0000","last_change":"2024-08-18T13:52:07.141608+0000","last_active":"2024-08-18T13:52:07.141636+0000","last_peered":"2024-08-18T13:52:07.141636+0000","last_clean":"2024-08-18T13:52:07.141636+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:52:07.141636+0000","last_undegraded":"2024-08-18T13:52:07.141636+0000","last_fullsized":"2024-08-18T13:52:07.141636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'67","last_scrub_stamp":"2024-08-18T13:52:07.141528+0000","last_deep_scrub":"54'67","last_deep_scrub_stamp":"2024-08-18T13:52:07.141528+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.141528+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:03.991301+0000","scrub_duration":0.018011553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":184,"num_write":164,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23450,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"54'714","reported_seq":2930,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.716447+0000","last_change":"2024-08-18T13:43:55.550748+0000","last_active":"2024-08-18T13:51:49.716447+0000","last_peered":"2024-08-18T13:51:49.716447+0000","last_clean":"2024-08-18T13:51:49.716447+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:51:49.716447+0000","last_undegraded":"2024-08-18T13:51:49.716447+0000","last_fullsized":"2024-08-18T13:51:49.716447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057306029999999999,"stat_sum":{"num_bytes":13242151,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":153,"num_read_kb":5341,"num_write":712,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.019415+0000","last_change":"2024-08-18T13:52:20.019359+0000","last_active":"2024-08-18T13:52:20.019415+0000","last_peered":"2024-08-18T13:52:20.019415+0000","last_clean":"2024-08-18T13:52:20.019415+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:52:20.019415+0000","last_undegraded":"2024-08-18T13:52:20.019415+0000","last_fullsized":"2024-08-18T13:52:20.019415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:20.019175+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:20.019175+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.019175+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:46:53.201543+0000","scrub_duration":0.0031831339999999998,"objects_trimmed":0,"snaptrim_duration":0.0064599770000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"54'292","reported_seq":351,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.937801+0000","last_change":"2024-08-18T13:52:16.937745+0000","last_active":"2024-08-18T13:52:16.937801+0000","last_peered":"2024-08-18T13:52:16.937801+0000","last_clean":"2024-08-18T13:52:16.937801+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:52:16.937801+0000","last_undegraded":"2024-08-18T13:52:16.937801+0000","last_fullsized":"2024-08-18T13:52:16.937801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'292","last_scrub_stamp":"2024-08-18T13:52:16.937598+0000","last_deep_scrub":"54'292","last_deep_scrub_stamp":"2024-08-18T13:52:16.937598+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.937598+0000","objects_scrubbed":3,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:46:56.453503+0000","scrub_duration":0.022315498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":12176,"num_write":284,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8544,"num_omap_keys":16,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"54'704","reported_seq":3426,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.674395+0000","last_change":"2024-08-18T13:43:56.151855+0000","last_active":"2024-08-18T13:51:49.674395+0000","last_peered":"2024-08-18T13:51:49.674395+0000","last_clean":"2024-08-18T13:51:49.674395+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:51:49.674395+0000","last_undegraded":"2024-08-18T13:51:49.674395+0000","last_fullsized":"2024-08-18T13:51:49.674395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060004689999999996,"stat_sum":{"num_bytes":28446730,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":659,"num_read_kb":21792,"num_write":702,"num_write_kb":28155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"54'84","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.470644+0000","last_change":"2024-08-18T13:52:21.470559+0000","last_active":"2024-08-18T13:52:21.470644+0000","last_peered":"2024-08-18T13:52:21.470644+0000","last_clean":"2024-08-18T13:52:21.470644+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:52:21.470644+0000","last_undegraded":"2024-08-18T13:52:21.470644+0000","last_fullsized":"2024-08-18T13:52:21.470644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'84","last_scrub_stamp":"2024-08-18T13:52:21.470358+0000","last_deep_scrub":"54'84","last_deep_scrub_stamp":"2024-08-18T13:52:21.470358+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.470358+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:32.524505+0000","scrub_duration":0.0039169649999999997,"objects_trimmed":0,"snaptrim_duration":0.006455484,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"54'660","reported_seq":714,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.030412+0000","last_change":"2024-08-18T13:52:16.030333+0000","last_active":"2024-08-18T13:52:16.030412+0000","last_peered":"2024-08-18T13:52:16.030412+0000","last_clean":"2024-08-18T13:52:16.030412+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:52:16.030412+0000","last_undegraded":"2024-08-18T13:52:16.030412+0000","last_fullsized":"2024-08-18T13:52:16.030412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'660","last_scrub_stamp":"2024-08-18T13:52:16.030100+0000","last_deep_scrub":"54'660","last_deep_scrub_stamp":"2024-08-18T13:52:16.030100+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.030100+0000","objects_scrubbed":3,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:55:53.439330+0000","scrub_duration":0.021515340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24518,"num_omap_keys":46,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"54'691","reported_seq":3367,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.103051+0000","last_change":"2024-08-18T13:52:31.102998+0000","last_active":"2024-08-18T13:52:31.103051+0000","last_peered":"2024-08-18T13:52:31.103051+0000","last_clean":"2024-08-18T13:52:31.103051+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:52:31.103051+0000","last_undegraded":"2024-08-18T13:52:31.103051+0000","last_fullsized":"2024-08-18T13:52:31.103051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'691","last_scrub_stamp":"2024-08-18T13:52:31.102865+0000","last_deep_scrub":"54'691","last_deep_scrub_stamp":"2024-08-18T13:52:31.102865+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.102865+0000","objects_scrubbed":341,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:16:32.971846+0000","scrub_duration":0.16053246500000001,"objects_trimmed":0,"snaptrim_duration":0.0067583870000000002,"stat_sum":{"num_bytes":38683931,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":590,"num_read_kb":16518,"num_write":694,"num_write_kb":42167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"54'97","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.632549+0000","last_change":"2024-08-18T13:52:08.632437+0000","last_active":"2024-08-18T13:52:08.632549+0000","last_peered":"2024-08-18T13:52:08.632549+0000","last_clean":"2024-08-18T13:52:08.632549+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:52:08.632549+0000","last_undegraded":"2024-08-18T13:52:08.632549+0000","last_fullsized":"2024-08-18T13:52:08.632549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'97","last_scrub_stamp":"2024-08-18T13:52:08.632220+0000","last_deep_scrub":"54'97","last_deep_scrub_stamp":"2024-08-18T13:52:08.632220+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.632220+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:30.186903+0000","scrub_duration":0.0038014519999999999,"objects_trimmed":0,"snaptrim_duration":0.005854756,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"54'387","reported_seq":453,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.408711+0000","last_change":"2024-08-18T13:52:09.408667+0000","last_active":"2024-08-18T13:52:09.408711+0000","last_peered":"2024-08-18T13:52:09.408711+0000","last_clean":"2024-08-18T13:52:09.408711+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:52:09.408711+0000","last_undegraded":"2024-08-18T13:52:09.408711+0000","last_fullsized":"2024-08-18T13:52:09.408711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'387","last_scrub_stamp":"2024-08-18T13:52:09.408576+0000","last_deep_scrub":"54'387","last_deep_scrub_stamp":"2024-08-18T13:52:09.408576+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.408576+0000","objects_scrubbed":4,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:46:58.406186+0000","scrub_duration":0.023922603000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":43191,"num_write":382,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16508,"num_omap_keys":31,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"54'661","reported_seq":4484,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:35.155057+0000","last_change":"2024-08-18T13:52:35.155004+0000","last_active":"2024-08-18T13:52:35.155057+0000","last_peered":"2024-08-18T13:52:35.155057+0000","last_clean":"2024-08-18T13:52:35.155057+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:52:35.155057+0000","last_undegraded":"2024-08-18T13:52:35.155057+0000","last_fullsized":"2024-08-18T13:52:35.155057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'661","last_scrub_stamp":"2024-08-18T13:52:35.154884+0000","last_deep_scrub":"54'661","last_deep_scrub_stamp":"2024-08-18T13:52:35.154884+0000","last_clean_scrub_stamp":"2024-08-18T13:52:35.154884+0000","objects_scrubbed":345,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:59:12.312487+0000","scrub_duration":0.140792578,"objects_trimmed":0,"snaptrim_duration":0.0067969390000000001,"stat_sum":{"num_bytes":51061167,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":1737,"num_read_kb":43040,"num_write":661,"num_write_kb":53481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"54'83","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.003797+0000","last_change":"2024-08-18T13:52:18.003732+0000","last_active":"2024-08-18T13:52:18.003797+0000","last_peered":"2024-08-18T13:52:18.003797+0000","last_clean":"2024-08-18T13:52:18.003797+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:52:18.003797+0000","last_undegraded":"2024-08-18T13:52:18.003797+0000","last_fullsized":"2024-08-18T13:52:18.003797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'83","last_scrub_stamp":"2024-08-18T13:52:18.003500+0000","last_deep_scrub":"54'83","last_deep_scrub_stamp":"2024-08-18T13:52:18.003500+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.003500+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:20.942863+0000","scrub_duration":0.0033910189999999999,"objects_trimmed":0,"snaptrim_duration":0.0063915389999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'122","reported_seq":296,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:11.436136+0000","last_change":"2024-08-18T13:52:11.436080+0000","last_active":"2024-08-18T13:52:11.436136+0000","last_peered":"2024-08-18T13:52:11.436136+0000","last_clean":"2024-08-18T13:52:11.436136+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:52:11.436136+0000","last_undegraded":"2024-08-18T13:52:11.436136+0000","last_fullsized":"2024-08-18T13:52:11.436136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'122","last_scrub_stamp":"2024-08-18T13:52:11.435881+0000","last_deep_scrub":"54'122","last_deep_scrub_stamp":"2024-08-18T13:52:11.435881+0000","last_clean_scrub_stamp":"2024-08-18T13:52:11.435881+0000","objects_scrubbed":2,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:53:26.277110+0000","scrub_duration":0.010000867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":638,"num_write":117,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14952,"num_omap_keys":28,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"54'643","reported_seq":3620,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:39.154461+0000","last_change":"2024-08-18T13:52:39.154377+0000","last_active":"2024-08-18T13:52:39.154461+0000","last_peered":"2024-08-18T13:52:39.154461+0000","last_clean":"2024-08-18T13:52:39.154461+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:52:39.154461+0000","last_undegraded":"2024-08-18T13:52:39.154461+0000","last_fullsized":"2024-08-18T13:52:39.154461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'643","last_scrub_stamp":"2024-08-18T13:52:39.154224+0000","last_deep_scrub":"54'643","last_deep_scrub_stamp":"2024-08-18T13:52:39.154224+0000","last_clean_scrub_stamp":"2024-08-18T13:52:39.154224+0000","objects_scrubbed":307,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:56:41.728995+0000","scrub_duration":0.12558661199999999,"objects_trimmed":0,"snaptrim_duration":0.0079242040000000007,"stat_sum":{"num_bytes":41790406,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":892,"num_read_kb":23319,"num_write":639,"num_write_kb":45248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'82","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:12.049564+0000","last_change":"2024-08-18T13:52:12.049490+0000","last_active":"2024-08-18T13:52:12.049564+0000","last_peered":"2024-08-18T13:52:12.049564+0000","last_clean":"2024-08-18T13:52:12.049564+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:52:12.049564+0000","last_undegraded":"2024-08-18T13:52:12.049564+0000","last_fullsized":"2024-08-18T13:52:12.049564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'82","last_scrub_stamp":"2024-08-18T13:52:12.049277+0000","last_deep_scrub":"54'82","last_deep_scrub_stamp":"2024-08-18T13:52:12.049277+0000","last_clean_scrub_stamp":"2024-08-18T13:52:12.049277+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:42:58.521893+0000","scrub_duration":0.0038715189999999999,"objects_trimmed":0,"snaptrim_duration":0.0041495459999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"54'240","reported_seq":307,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:13.957043+0000","last_change":"2024-08-18T13:52:13.956993+0000","last_active":"2024-08-18T13:52:13.957043+0000","last_peered":"2024-08-18T13:52:13.957043+0000","last_clean":"2024-08-18T13:52:13.957043+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:52:13.957043+0000","last_undegraded":"2024-08-18T13:52:13.957043+0000","last_fullsized":"2024-08-18T13:52:13.957043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'240","last_scrub_stamp":"2024-08-18T13:52:13.956898+0000","last_deep_scrub":"54'240","last_deep_scrub_stamp":"2024-08-18T13:52:13.956898+0000","last_clean_scrub_stamp":"2024-08-18T13:52:13.956898+0000","objects_scrubbed":6,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:30:29.246162+0000","scrub_duration":0.0255698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":49152,"num_write":231,"num_write_kb":12421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24290,"num_omap_keys":46,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"54'716","reported_seq":3241,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:32.135169+0000","last_change":"2024-08-18T13:52:32.135117+0000","last_active":"2024-08-18T13:52:32.135169+0000","last_peered":"2024-08-18T13:52:32.135169+0000","last_clean":"2024-08-18T13:52:32.135169+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:52:32.135169+0000","last_undegraded":"2024-08-18T13:52:32.135169+0000","last_fullsized":"2024-08-18T13:52:32.135169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'716","last_scrub_stamp":"2024-08-18T13:52:32.134994+0000","last_deep_scrub":"54'716","last_deep_scrub_stamp":"2024-08-18T13:52:32.134994+0000","last_clean_scrub_stamp":"2024-08-18T13:52:32.134994+0000","objects_scrubbed":352,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:10.068446+0000","scrub_duration":0.13071303000000001,"objects_trimmed":0,"snaptrim_duration":0.0058974379999999996,"stat_sum":{"num_bytes":30462084,"num_objects":352,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":439,"num_read_kb":15189,"num_write":710,"num_write_kb":30067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"54'90","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.918324+0000","last_change":"2024-08-18T13:52:09.918219+0000","last_active":"2024-08-18T13:52:09.918324+0000","last_peered":"2024-08-18T13:52:09.918324+0000","last_clean":"2024-08-18T13:52:09.918324+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:52:09.918324+0000","last_undegraded":"2024-08-18T13:52:09.918324+0000","last_fullsized":"2024-08-18T13:52:09.918324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:09.917892+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:09.917892+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.917892+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:20.972039+0000","scrub_duration":0.0039382109999999996,"objects_trimmed":0,"snaptrim_duration":0.0068182370000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"54'983","reported_seq":1039,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.478599+0000","last_change":"2024-08-18T13:52:07.478553+0000","last_active":"2024-08-18T13:52:07.478599+0000","last_peered":"2024-08-18T13:52:07.478599+0000","last_clean":"2024-08-18T13:52:07.478599+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:52:07.478599+0000","last_undegraded":"2024-08-18T13:52:07.478599+0000","last_fullsized":"2024-08-18T13:52:07.478599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'983","last_scrub_stamp":"2024-08-18T13:52:07.478392+0000","last_deep_scrub":"54'983","last_deep_scrub_stamp":"2024-08-18T13:52:07.478392+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.478392+0000","objects_scrubbed":2,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:52:04.274947+0000","scrub_duration":0.015274658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2087,"num_write":979,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8826,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"54'119","reported_seq":208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:06.906625+0000","last_change":"2024-08-18T13:52:06.906561+0000","last_active":"2024-08-18T13:52:06.906625+0000","last_peered":"2024-08-18T13:52:06.906625+0000","last_clean":"2024-08-18T13:52:06.906625+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:52:06.906625+0000","last_undegraded":"2024-08-18T13:52:06.906625+0000","last_fullsized":"2024-08-18T13:52:06.906625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'119","last_scrub_stamp":"2024-08-18T13:52:06.906373+0000","last_deep_scrub":"54'119","last_deep_scrub_stamp":"2024-08-18T13:52:06.906373+0000","last_clean_scrub_stamp":"2024-08-18T13:52:06.906373+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:57:50.221813+0000","scrub_duration":0.0082944209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"54'664","reported_seq":3549,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:32.413005+0000","last_change":"2024-08-18T13:52:32.412930+0000","last_active":"2024-08-18T13:52:32.413005+0000","last_peered":"2024-08-18T13:52:32.413005+0000","last_clean":"2024-08-18T13:52:32.413005+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:52:32.413005+0000","last_undegraded":"2024-08-18T13:52:32.413005+0000","last_fullsized":"2024-08-18T13:52:32.413005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'664","last_scrub_stamp":"2024-08-18T13:52:32.412772+0000","last_deep_scrub":"54'664","last_deep_scrub_stamp":"2024-08-18T13:52:32.412772+0000","last_clean_scrub_stamp":"2024-08-18T13:52:32.412772+0000","objects_scrubbed":364,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:29:53.313639+0000","scrub_duration":0.15394601399999999,"objects_trimmed":0,"snaptrim_duration":0.0035684480000000001,"stat_sum":{"num_bytes":39930170,"num_objects":364,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":798,"num_read_kb":26403,"num_write":662,"num_write_kb":47533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:24.899358+0000","last_change":"2024-08-18T13:52:24.899284+0000","last_active":"2024-08-18T13:52:24.899358+0000","last_peered":"2024-08-18T13:52:24.899358+0000","last_clean":"2024-08-18T13:52:24.899358+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:52:24.899358+0000","last_undegraded":"2024-08-18T13:52:24.899358+0000","last_fullsized":"2024-08-18T13:52:24.899358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:24.899079+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:24.899079+0000","last_clean_scrub_stamp":"2024-08-18T13:52:24.899079+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:46:27.076634+0000","scrub_duration":0.0037898570000000002,"objects_trimmed":0,"snaptrim_duration":0.0057711639999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"54'80","reported_seq":133,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.495618+0000","last_change":"2024-08-18T13:52:16.495514+0000","last_active":"2024-08-18T13:52:16.495618+0000","last_peered":"2024-08-18T13:52:16.495618+0000","last_clean":"2024-08-18T13:52:16.495618+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:52:16.495618+0000","last_undegraded":"2024-08-18T13:52:16.495618+0000","last_fullsized":"2024-08-18T13:52:16.495618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'80","last_scrub_stamp":"2024-08-18T13:52:16.495259+0000","last_deep_scrub":"54'80","last_deep_scrub_stamp":"2024-08-18T13:52:16.495259+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.495259+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:47:37.229120+0000","scrub_duration":0.016550458000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31498,"num_omap_keys":60,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"54'688","reported_seq":4151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.418732+0000","last_change":"2024-08-18T13:52:31.418655+0000","last_active":"2024-08-18T13:52:31.418732+0000","last_peered":"2024-08-18T13:52:31.418732+0000","last_clean":"2024-08-18T13:52:31.418732+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:52:31.418732+0000","last_undegraded":"2024-08-18T13:52:31.418732+0000","last_fullsized":"2024-08-18T13:52:31.418732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'688","last_scrub_stamp":"2024-08-18T13:52:31.418510+0000","last_deep_scrub":"54'688","last_deep_scrub_stamp":"2024-08-18T13:52:31.418510+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.418510+0000","objects_scrubbed":349,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:54:04.889299+0000","scrub_duration":0.17638959800000001,"objects_trimmed":0,"snaptrim_duration":0.0032433420000000002,"stat_sum":{"num_bytes":54557211,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1377,"num_read_kb":50055,"num_write":687,"num_write_kb":57734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"54'95","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.530780+0000","last_change":"2024-08-18T13:52:27.530717+0000","last_active":"2024-08-18T13:52:27.530780+0000","last_peered":"2024-08-18T13:52:27.530780+0000","last_clean":"2024-08-18T13:52:27.530780+0000","last_became_active":"2024-08-18T13:40:16.614814+0000","last_became_peered":"2024-08-18T13:40:16.614814+0000","last_unstale":"2024-08-18T13:52:27.530780+0000","last_undegraded":"2024-08-18T13:52:27.530780+0000","last_fullsized":"2024-08-18T13:52:27.530780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'95","last_scrub_stamp":"2024-08-18T13:52:27.530504+0000","last_deep_scrub":"54'95","last_deep_scrub_stamp":"2024-08-18T13:52:27.530504+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.530504+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:13.569364+0000","scrub_duration":0.003176417,"objects_trimmed":0,"snaptrim_duration":0.0033975450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'36","reported_seq":101,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.170255+0000","last_change":"2024-08-18T13:52:10.170197+0000","last_active":"2024-08-18T13:52:10.170255+0000","last_peered":"2024-08-18T13:52:10.170255+0000","last_clean":"2024-08-18T13:52:10.170255+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:52:10.170255+0000","last_undegraded":"2024-08-18T13:52:10.170255+0000","last_fullsized":"2024-08-18T13:52:10.170255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'36","last_scrub_stamp":"2024-08-18T13:52:10.170013+0000","last_deep_scrub":"54'36","last_deep_scrub_stamp":"2024-08-18T13:52:10.170013+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.170013+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:21.224160+0000","scrub_duration":0.017405436999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":33,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24938,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"54'670","reported_seq":3235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.683385+0000","last_change":"2024-08-18T13:43:56.153281+0000","last_active":"2024-08-18T13:51:49.683385+0000","last_peered":"2024-08-18T13:51:49.683385+0000","last_clean":"2024-08-18T13:51:49.683385+0000","last_became_active":"2024-08-18T13:40:16.609908+0000","last_became_peered":"2024-08-18T13:40:16.609908+0000","last_unstale":"2024-08-18T13:51:49.683385+0000","last_undegraded":"2024-08-18T13:51:49.683385+0000","last_fullsized":"2024-08-18T13:51:49.683385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069871079999999997,"stat_sum":{"num_bytes":26719276,"num_objects":358,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":491,"num_read_kb":18148,"num_write":671,"num_write_kb":26414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4,5],"acting":[0,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'87","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:32.867441+0000","last_change":"2024-08-18T13:52:32.867312+0000","last_active":"2024-08-18T13:52:32.867441+0000","last_peered":"2024-08-18T13:52:32.867441+0000","last_clean":"2024-08-18T13:52:32.867441+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:52:32.867441+0000","last_undegraded":"2024-08-18T13:52:32.867441+0000","last_fullsized":"2024-08-18T13:52:32.867441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:32.867113+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:32.867113+0000","last_clean_scrub_stamp":"2024-08-18T13:52:32.867113+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:29:53.767979+0000","scrub_duration":0.003763452,"objects_trimmed":0,"snaptrim_duration":0.0057951929999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"54'32","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.525558+0000","last_change":"2024-08-18T13:52:18.525490+0000","last_active":"2024-08-18T13:52:18.525558+0000","last_peered":"2024-08-18T13:52:18.525558+0000","last_clean":"2024-08-18T13:52:18.525558+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:52:18.525558+0000","last_undegraded":"2024-08-18T13:52:18.525558+0000","last_fullsized":"2024-08-18T13:52:18.525558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'32","last_scrub_stamp":"2024-08-18T13:52:18.525224+0000","last_deep_scrub":"54'32","last_deep_scrub_stamp":"2024-08-18T13:52:18.525224+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.525224+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:52:15.321780+0000","scrub_duration":0.016102814999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":30,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9612,"num_omap_keys":18,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"54'688","reported_seq":2842,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.583212+0000","last_change":"2024-08-18T13:52:34.583104+0000","last_active":"2024-08-18T13:52:34.583212+0000","last_peered":"2024-08-18T13:52:34.583212+0000","last_clean":"2024-08-18T13:52:34.583212+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:52:34.583212+0000","last_undegraded":"2024-08-18T13:52:34.583212+0000","last_fullsized":"2024-08-18T13:52:34.583212+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'688","last_scrub_stamp":"2024-08-18T13:52:34.582985+0000","last_deep_scrub":"54'688","last_deep_scrub_stamp":"2024-08-18T13:52:34.582985+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.582985+0000","objects_scrubbed":346,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:53:19.774572+0000","scrub_duration":0.112945194,"objects_trimmed":0,"snaptrim_duration":0.0070733389999999997,"stat_sum":{"num_bytes":17282460,"num_objects":346,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":68,"num_read_kb":8224,"num_write":686,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"54'66","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.910326+0000","last_change":"2024-08-18T13:52:04.910259+0000","last_active":"2024-08-18T13:52:04.910326+0000","last_peered":"2024-08-18T13:52:04.910326+0000","last_clean":"2024-08-18T13:52:04.910326+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:52:04.910326+0000","last_undegraded":"2024-08-18T13:52:04.910326+0000","last_fullsized":"2024-08-18T13:52:04.910326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'66","last_scrub_stamp":"2024-08-18T13:52:04.910054+0000","last_deep_scrub":"54'66","last_deep_scrub_stamp":"2024-08-18T13:52:04.910054+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.910054+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:14:14.576424+0000","scrub_duration":0.0032501320000000002,"objects_trimmed":0,"snaptrim_duration":0.0060906930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"54'29","reported_seq":94,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.600601+0000","last_change":"2024-08-18T13:52:04.600549+0000","last_active":"2024-08-18T13:52:04.600601+0000","last_peered":"2024-08-18T13:52:04.600601+0000","last_clean":"2024-08-18T13:52:04.600601+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:52:04.600601+0000","last_undegraded":"2024-08-18T13:52:04.600601+0000","last_fullsized":"2024-08-18T13:52:04.600601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'29","last_scrub_stamp":"2024-08-18T13:52:04.600439+0000","last_deep_scrub":"54'29","last_deep_scrub_stamp":"2024-08-18T13:52:04.600439+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.600439+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:26.017443+0000","scrub_duration":0.014861572999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":27,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"54'695","reported_seq":3654,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:33.838023+0000","last_change":"2024-08-18T13:52:33.837968+0000","last_active":"2024-08-18T13:52:33.838023+0000","last_peered":"2024-08-18T13:52:33.838023+0000","last_clean":"2024-08-18T13:52:33.838023+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:52:33.838023+0000","last_undegraded":"2024-08-18T13:52:33.838023+0000","last_fullsized":"2024-08-18T13:52:33.838023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'695","last_scrub_stamp":"2024-08-18T13:52:33.837813+0000","last_deep_scrub":"54'695","last_deep_scrub_stamp":"2024-08-18T13:52:33.837813+0000","last_clean_scrub_stamp":"2024-08-18T13:52:33.837813+0000","objects_scrubbed":345,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:56:26.045133+0000","scrub_duration":0.159429342,"objects_trimmed":0,"snaptrim_duration":0.011606655,"stat_sum":{"num_bytes":44119789,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":873,"num_read_kb":37157,"num_write":696,"num_write_kb":47501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"54'91","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.858350+0000","last_change":"2024-08-18T13:52:30.858279+0000","last_active":"2024-08-18T13:52:30.858350+0000","last_peered":"2024-08-18T13:52:30.858350+0000","last_clean":"2024-08-18T13:52:30.858350+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:52:30.858350+0000","last_undegraded":"2024-08-18T13:52:30.858350+0000","last_fullsized":"2024-08-18T13:52:30.858350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:30.858089+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:30.858089+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.858089+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:32:05.996511+0000","scrub_duration":0.0036829689999999999,"objects_trimmed":0,"snaptrim_duration":0.0053019490000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"54'1212","reported_seq":1295,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.493306+0000","last_change":"2024-08-18T13:52:14.493275+0000","last_active":"2024-08-18T13:52:14.493306+0000","last_peered":"2024-08-18T13:52:14.493306+0000","last_clean":"2024-08-18T13:52:14.493306+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:52:14.493306+0000","last_undegraded":"2024-08-18T13:52:14.493306+0000","last_fullsized":"2024-08-18T13:52:14.493306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'1212","last_scrub_stamp":"2024-08-18T13:52:14.493191+0000","last_deep_scrub":"54'1212","last_deep_scrub_stamp":"2024-08-18T13:52:14.493191+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.493191+0000","objects_scrubbed":8,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:29:04.116014+0000","scrub_duration":0.026335695999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":78,"num_read_kb":12399,"num_write":1202,"num_write_kb":8830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18540,"num_omap_keys":36,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'657","reported_seq":3538,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:33.123467+0000","last_change":"2024-08-18T13:52:33.123383+0000","last_active":"2024-08-18T13:52:33.123467+0000","last_peered":"2024-08-18T13:52:33.123467+0000","last_clean":"2024-08-18T13:52:33.123467+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:52:33.123467+0000","last_undegraded":"2024-08-18T13:52:33.123467+0000","last_fullsized":"2024-08-18T13:52:33.123467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'657","last_scrub_stamp":"2024-08-18T13:52:33.123217+0000","last_deep_scrub":"54'657","last_deep_scrub_stamp":"2024-08-18T13:52:33.123217+0000","last_clean_scrub_stamp":"2024-08-18T13:52:33.123217+0000","objects_scrubbed":296,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:08.246627+0000","scrub_duration":0.120851848,"objects_trimmed":0,"snaptrim_duration":0.0054104280000000001,"stat_sum":{"num_bytes":33280018,"num_objects":296,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296,"num_whiteouts":0,"num_read":797,"num_read_kb":22024,"num_write":655,"num_write_kb":32783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.315208+0000","last_change":"2024-08-18T13:52:18.315124+0000","last_active":"2024-08-18T13:52:18.315208+0000","last_peered":"2024-08-18T13:52:18.315208+0000","last_clean":"2024-08-18T13:52:18.315208+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:52:18.315208+0000","last_undegraded":"2024-08-18T13:52:18.315208+0000","last_fullsized":"2024-08-18T13:52:18.315208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:18.314906+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:18.314906+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.314906+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:39.731910+0000","scrub_duration":0.0038296580000000001,"objects_trimmed":0,"snaptrim_duration":0.0044255090000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"54'8","reported_seq":61,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.301096+0000","last_change":"2024-08-18T13:52:09.301035+0000","last_active":"2024-08-18T13:52:09.301096+0000","last_peered":"2024-08-18T13:52:09.301096+0000","last_clean":"2024-08-18T13:52:09.301096+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:52:09.301096+0000","last_undegraded":"2024-08-18T13:52:09.301096+0000","last_fullsized":"2024-08-18T13:52:09.301096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'8","last_scrub_stamp":"2024-08-18T13:52:09.300822+0000","last_deep_scrub":"54'8","last_deep_scrub_stamp":"2024-08-18T13:52:09.300822+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.300822+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:39:45.416911+0000","scrub_duration":0.008909953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28532,"num_omap_keys":54,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"54'660","reported_seq":2780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.749372+0000","last_change":"2024-08-18T13:43:55.555242+0000","last_active":"2024-08-18T13:51:49.749372+0000","last_peered":"2024-08-18T13:51:49.749372+0000","last_clean":"2024-08-18T13:51:49.749372+0000","last_became_active":"2024-08-18T13:40:16.915369+0000","last_became_peered":"2024-08-18T13:40:16.915369+0000","last_unstale":"2024-08-18T13:51:49.749372+0000","last_undegraded":"2024-08-18T13:51:49.749372+0000","last_fullsized":"2024-08-18T13:51:49.749372+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005619059,"stat_sum":{"num_bytes":16977484,"num_objects":353,"num_object_clones":0,"num_object_copies":1412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":46,"num_read_kb":4277,"num_write":660,"num_write_kb":20975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,6],"acting":[3,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"54'101","reported_seq":2175,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:25.870786+0000","last_change":"2024-08-18T13:52:25.870697+0000","last_active":"2024-08-18T13:52:25.870786+0000","last_peered":"2024-08-18T13:52:25.870786+0000","last_clean":"2024-08-18T13:52:25.870786+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:52:25.870786+0000","last_undegraded":"2024-08-18T13:52:25.870786+0000","last_fullsized":"2024-08-18T13:52:25.870786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'101","last_scrub_stamp":"2024-08-18T13:52:25.870489+0000","last_deep_scrub":"54'101","last_deep_scrub_stamp":"2024-08-18T13:52:25.870489+0000","last_clean_scrub_stamp":"2024-08-18T13:52:25.870489+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:19:17.425779+0000","scrub_duration":0.0032600789999999999,"objects_trimmed":0,"snaptrim_duration":0.0064019180000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"54'4","reported_seq":58,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.395883+0000","last_change":"2024-08-18T13:52:15.395814+0000","last_active":"2024-08-18T13:52:15.395883+0000","last_peered":"2024-08-18T13:52:15.395883+0000","last_clean":"2024-08-18T13:52:15.395883+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:52:15.395883+0000","last_undegraded":"2024-08-18T13:52:15.395883+0000","last_fullsized":"2024-08-18T13:52:15.395883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'4","last_scrub_stamp":"2024-08-18T13:52:15.395609+0000","last_deep_scrub":"54'4","last_deep_scrub_stamp":"2024-08-18T13:52:15.395609+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.395609+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:46.408855+0000","scrub_duration":0.0073810990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15112,"num_omap_keys":29,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"54'625","reported_seq":3607,"reported_epoch":54,"state":"active+clean+scrubbing+deep","last_fresh":"2024-08-18T13:52:41.293042+0000","last_change":"2024-08-18T13:52:41.261155+0000","last_active":"2024-08-18T13:52:41.293042+0000","last_peered":"2024-08-18T13:52:41.293042+0000","last_clean":"2024-08-18T13:52:41.293042+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:52:41.293042+0000","last_undegraded":"2024-08-18T13:52:41.293042+0000","last_fullsized":"2024-08-18T13:52:41.293042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":99,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074125900000000002,"stat_sum":{"num_bytes":40693231,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":911,"num_read_kb":37155,"num_write":622,"num_write_kb":48255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.056426+0000","last_change":"2024-08-18T13:52:08.056345+0000","last_active":"2024-08-18T13:52:08.056426+0000","last_peered":"2024-08-18T13:52:08.056426+0000","last_clean":"2024-08-18T13:52:08.056426+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:52:08.056426+0000","last_undegraded":"2024-08-18T13:52:08.056426+0000","last_fullsized":"2024-08-18T13:52:08.056426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:08.056130+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:08.056130+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.056130+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:54.094989+0000","scrub_duration":0.003803248,"objects_trimmed":0,"snaptrim_duration":0.0059765870000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"54'220","reported_seq":298,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:02.096320+0000","last_change":"2024-08-18T13:52:02.096276+0000","last_active":"2024-08-18T13:52:02.096320+0000","last_peered":"2024-08-18T13:52:02.096320+0000","last_clean":"2024-08-18T13:52:02.096320+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:52:02.096320+0000","last_undegraded":"2024-08-18T13:52:02.096320+0000","last_fullsized":"2024-08-18T13:52:02.096320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'220","last_scrub_stamp":"2024-08-18T13:52:02.096123+0000","last_deep_scrub":"54'220","last_deep_scrub_stamp":"2024-08-18T13:52:02.096123+0000","last_clean_scrub_stamp":"2024-08-18T13:52:02.096123+0000","objects_scrubbed":6,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:09.274537+0000","scrub_duration":0.023746768000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":45098,"num_write":215,"num_write_kb":8331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11822,"num_omap_keys":23,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'629","reported_seq":3964,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:39.478236+0000","last_change":"2024-08-18T13:52:39.478158+0000","last_active":"2024-08-18T13:52:39.478236+0000","last_peered":"2024-08-18T13:52:39.478236+0000","last_clean":"2024-08-18T13:52:39.478236+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:52:39.478236+0000","last_undegraded":"2024-08-18T13:52:39.478236+0000","last_fullsized":"2024-08-18T13:52:39.478236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'629","last_scrub_stamp":"2024-08-18T13:52:39.478005+0000","last_deep_scrub":"54'629","last_deep_scrub_stamp":"2024-08-18T13:52:39.478005+0000","last_clean_scrub_stamp":"2024-08-18T13:52:39.478005+0000","objects_scrubbed":306,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:48.229574+0000","scrub_duration":0.147969562,"objects_trimmed":0,"snaptrim_duration":0.0084356780000000003,"stat_sum":{"num_bytes":50488256,"num_objects":306,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":1250,"num_read_kb":43248,"num_write":629,"num_write_kb":49603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'84","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:26.838724+0000","last_change":"2024-08-18T13:52:26.838660+0000","last_active":"2024-08-18T13:52:26.838724+0000","last_peered":"2024-08-18T13:52:26.838724+0000","last_clean":"2024-08-18T13:52:26.838724+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:52:26.838724+0000","last_undegraded":"2024-08-18T13:52:26.838724+0000","last_fullsized":"2024-08-18T13:52:26.838724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'84","last_scrub_stamp":"2024-08-18T13:52:26.838465+0000","last_deep_scrub":"54'84","last_deep_scrub_stamp":"2024-08-18T13:52:26.838465+0000","last_clean_scrub_stamp":"2024-08-18T13:52:26.838465+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:59:19.259738+0000","scrub_duration":0.003882019,"objects_trimmed":0,"snaptrim_duration":0.0065977270000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"54'241","reported_seq":325,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:12.244378+0000","last_change":"2024-08-18T13:52:12.244303+0000","last_active":"2024-08-18T13:52:12.244378+0000","last_peered":"2024-08-18T13:52:12.244378+0000","last_clean":"2024-08-18T13:52:12.244378+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:52:12.244378+0000","last_undegraded":"2024-08-18T13:52:12.244378+0000","last_fullsized":"2024-08-18T13:52:12.244378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'240","last_scrub_stamp":"2024-08-18T13:52:12.244185+0000","last_deep_scrub":"54'240","last_deep_scrub_stamp":"2024-08-18T13:52:12.244185+0000","last_clean_scrub_stamp":"2024-08-18T13:52:12.244185+0000","objects_scrubbed":8,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:30:27.533449+0000","scrub_duration":0.029778697999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583572,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":53278,"num_write":227,"num_write_kb":12433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22962,"num_omap_keys":43,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"54'671","reported_seq":3286,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.098343+0000","last_change":"2024-08-18T13:52:30.098271+0000","last_active":"2024-08-18T13:52:30.098343+0000","last_peered":"2024-08-18T13:52:30.098343+0000","last_clean":"2024-08-18T13:52:30.098343+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:52:30.098343+0000","last_undegraded":"2024-08-18T13:52:30.098343+0000","last_fullsized":"2024-08-18T13:52:30.098343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'671","last_scrub_stamp":"2024-08-18T13:52:30.098121+0000","last_deep_scrub":"54'671","last_deep_scrub_stamp":"2024-08-18T13:52:30.098121+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.098121+0000","objects_scrubbed":343,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:05.221530+0000","scrub_duration":0.130780905,"objects_trimmed":0,"snaptrim_duration":0.0076714670000000004,"stat_sum":{"num_bytes":16086174,"num_objects":343,"num_object_clones":0,"num_object_copies":1372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":529,"num_read_kb":12373,"num_write":668,"num_write_kb":24206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"54'86","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.670616+0000","last_change":"2024-08-18T13:52:15.670550+0000","last_active":"2024-08-18T13:52:15.670616+0000","last_peered":"2024-08-18T13:52:15.670616+0000","last_clean":"2024-08-18T13:52:15.670616+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:52:15.670616+0000","last_undegraded":"2024-08-18T13:52:15.670616+0000","last_fullsized":"2024-08-18T13:52:15.670616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'86","last_scrub_stamp":"2024-08-18T13:52:15.670343+0000","last_deep_scrub":"54'86","last_deep_scrub_stamp":"2024-08-18T13:52:15.670343+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.670343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:49.587682+0000","scrub_duration":0.0034242740000000002,"objects_trimmed":0,"snaptrim_duration":0.0039212459999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"54'173","reported_seq":242,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.045210+0000","last_change":"2024-08-18T13:52:14.045155+0000","last_active":"2024-08-18T13:52:14.045210+0000","last_peered":"2024-08-18T13:52:14.045210+0000","last_clean":"2024-08-18T13:52:14.045210+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:52:14.045210+0000","last_undegraded":"2024-08-18T13:52:14.045210+0000","last_fullsized":"2024-08-18T13:52:14.045210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'173","last_scrub_stamp":"2024-08-18T13:52:14.044993+0000","last_deep_scrub":"54'173","last_deep_scrub_stamp":"2024-08-18T13:52:14.044993+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.044993+0000","objects_scrubbed":4,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:39.376618+0000","scrub_duration":0.013861017999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8216,"num_write":168,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'644","reported_seq":3302,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:40.726423+0000","last_change":"2024-08-18T13:52:40.726351+0000","last_active":"2024-08-18T13:52:40.726423+0000","last_peered":"2024-08-18T13:52:40.726423+0000","last_clean":"2024-08-18T13:52:40.726423+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:52:40.726423+0000","last_undegraded":"2024-08-18T13:52:40.726423+0000","last_fullsized":"2024-08-18T13:52:40.726423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'644","last_scrub_stamp":"2024-08-18T13:52:40.726235+0000","last_deep_scrub":"54'644","last_deep_scrub_stamp":"2024-08-18T13:52:40.726235+0000","last_clean_scrub_stamp":"2024-08-18T13:52:40.726235+0000","objects_scrubbed":325,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:48:53.529215+0000","scrub_duration":0.12138481800000001,"objects_trimmed":0,"snaptrim_duration":0.0059375670000000004,"stat_sum":{"num_bytes":15740586,"num_objects":325,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":572,"num_read_kb":18359,"num_write":643,"num_write_kb":23895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"54'89","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.002384+0000","last_change":"2024-08-18T13:52:14.002238+0000","last_active":"2024-08-18T13:52:14.002384+0000","last_peered":"2024-08-18T13:52:14.002384+0000","last_clean":"2024-08-18T13:52:14.002384+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:52:14.002384+0000","last_undegraded":"2024-08-18T13:52:14.002384+0000","last_fullsized":"2024-08-18T13:52:14.002384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'89","last_scrub_stamp":"2024-08-18T13:52:14.002012+0000","last_deep_scrub":"54'89","last_deep_scrub_stamp":"2024-08-18T13:52:14.002012+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.002012+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:43.739908+0000","scrub_duration":0.0036712979999999999,"objects_trimmed":0,"snaptrim_duration":0.0057226309999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"54'237","reported_seq":297,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.513342+0000","last_change":"2024-08-18T13:52:05.513291+0000","last_active":"2024-08-18T13:52:05.513342+0000","last_peered":"2024-08-18T13:52:05.513342+0000","last_clean":"2024-08-18T13:52:05.513342+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:52:05.513342+0000","last_undegraded":"2024-08-18T13:52:05.513342+0000","last_fullsized":"2024-08-18T13:52:05.513342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'237","last_scrub_stamp":"2024-08-18T13:52:05.513154+0000","last_deep_scrub":"54'237","last_deep_scrub_stamp":"2024-08-18T13:52:05.513154+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.513154+0000","objects_scrubbed":4,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:47:26.247014+0000","scrub_duration":0.018778702000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":20397,"num_write":232,"num_write_kb":4254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17234,"num_omap_keys":33,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"54'677","reported_seq":3170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.525636+0000","last_change":"2024-08-18T13:43:55.553346+0000","last_active":"2024-08-18T13:51:49.525636+0000","last_peered":"2024-08-18T13:51:49.525636+0000","last_clean":"2024-08-18T13:51:49.525636+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:51:49.525636+0000","last_undegraded":"2024-08-18T13:51:49.525636+0000","last_fullsized":"2024-08-18T13:51:49.525636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073620359999999998,"stat_sum":{"num_bytes":31240743,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":431,"num_read_kb":13577,"num_write":670,"num_write_kb":34133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"54'106","reported_seq":2181,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.455857+0000","last_change":"2024-08-18T13:52:18.455745+0000","last_active":"2024-08-18T13:52:18.455857+0000","last_peered":"2024-08-18T13:52:18.455857+0000","last_clean":"2024-08-18T13:52:18.455857+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:52:18.455857+0000","last_undegraded":"2024-08-18T13:52:18.455857+0000","last_fullsized":"2024-08-18T13:52:18.455857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'106","last_scrub_stamp":"2024-08-18T13:52:18.455540+0000","last_deep_scrub":"54'106","last_deep_scrub_stamp":"2024-08-18T13:52:18.455540+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.455540+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:15.305313+0000","scrub_duration":0.003717709,"objects_trimmed":0,"snaptrim_duration":0.0059613369999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"54'14","reported_seq":67,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.267260+0000","last_change":"2024-08-18T13:52:14.267230+0000","last_active":"2024-08-18T13:52:14.267260+0000","last_peered":"2024-08-18T13:52:14.267260+0000","last_clean":"2024-08-18T13:52:14.267260+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:52:14.267260+0000","last_undegraded":"2024-08-18T13:52:14.267260+0000","last_fullsized":"2024-08-18T13:52:14.267260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'14","last_scrub_stamp":"2024-08-18T13:52:14.267137+0000","last_deep_scrub":"54'14","last_deep_scrub_stamp":"2024-08-18T13:52:14.267137+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.267137+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:59.683518+0000","scrub_duration":0.010501551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":39082,"num_omap_keys":74,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"54'699","reported_seq":3481,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.677703+0000","last_change":"2024-08-18T13:43:55.553902+0000","last_active":"2024-08-18T13:51:49.677703+0000","last_peered":"2024-08-18T13:51:49.677703+0000","last_clean":"2024-08-18T13:51:49.677703+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:51:49.677703+0000","last_undegraded":"2024-08-18T13:51:49.677703+0000","last_fullsized":"2024-08-18T13:51:49.677703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076808309999999999,"stat_sum":{"num_bytes":29636158,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":719,"num_read_kb":24704,"num_write":698,"num_write_kb":38783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"54'650","reported_seq":3316,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.124164+0000","last_change":"2024-08-18T13:52:34.124106+0000","last_active":"2024-08-18T13:52:34.124164+0000","last_peered":"2024-08-18T13:52:34.124164+0000","last_clean":"2024-08-18T13:52:34.124164+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:52:34.124164+0000","last_undegraded":"2024-08-18T13:52:34.124164+0000","last_fullsized":"2024-08-18T13:52:34.124164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'650","last_scrub_stamp":"2024-08-18T13:52:34.123992+0000","last_deep_scrub":"54'650","last_deep_scrub_stamp":"2024-08-18T13:52:34.123992+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.123992+0000","objects_scrubbed":334,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:16:35.992973+0000","scrub_duration":0.12763080500000001,"objects_trimmed":0,"snaptrim_duration":0.0068864499999999997,"stat_sum":{"num_bytes":24960867,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":580,"num_read_kb":20952,"num_write":650,"num_write_kb":24715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"54'91","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.205937+0000","last_change":"2024-08-18T13:52:05.205870+0000","last_active":"2024-08-18T13:52:05.205937+0000","last_peered":"2024-08-18T13:52:05.205937+0000","last_clean":"2024-08-18T13:52:05.205937+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:52:05.205937+0000","last_undegraded":"2024-08-18T13:52:05.205937+0000","last_fullsized":"2024-08-18T13:52:05.205937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:05.205644+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:05.205644+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.205644+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:51.244504+0000","scrub_duration":0.0033367140000000002,"objects_trimmed":0,"snaptrim_duration":0.0066095759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"54'203","reported_seq":267,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.395398+0000","last_change":"2024-08-18T13:52:10.395341+0000","last_active":"2024-08-18T13:52:10.395398+0000","last_peered":"2024-08-18T13:52:10.395398+0000","last_clean":"2024-08-18T13:52:10.395398+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:52:10.395398+0000","last_undegraded":"2024-08-18T13:52:10.395398+0000","last_fullsized":"2024-08-18T13:52:10.395398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'203","last_scrub_stamp":"2024-08-18T13:52:10.395155+0000","last_deep_scrub":"54'203","last_deep_scrub_stamp":"2024-08-18T13:52:10.395155+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.395155+0000","objects_scrubbed":6,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:17.573569+0000","scrub_duration":0.030766387999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":197,"num_write_kb":12441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25166,"num_omap_keys":49,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"54'658","reported_seq":3211,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.475022+0000","last_change":"2024-08-18T13:52:34.474804+0000","last_active":"2024-08-18T13:52:34.475022+0000","last_peered":"2024-08-18T13:52:34.475022+0000","last_clean":"2024-08-18T13:52:34.475022+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:52:34.475022+0000","last_undegraded":"2024-08-18T13:52:34.475022+0000","last_fullsized":"2024-08-18T13:52:34.475022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'658","last_scrub_stamp":"2024-08-18T13:52:34.474541+0000","last_deep_scrub":"54'658","last_deep_scrub_stamp":"2024-08-18T13:52:34.474541+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.474541+0000","objects_scrubbed":338,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:49.261380+0000","scrub_duration":0.15356631400000001,"objects_trimmed":0,"snaptrim_duration":0.0047613839999999996,"stat_sum":{"num_bytes":20867535,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":467,"num_read_kb":16390,"num_write":656,"num_write_kb":20726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":2183,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:33.293895+0000","last_change":"2024-08-18T13:52:33.293834+0000","last_active":"2024-08-18T13:52:33.293895+0000","last_peered":"2024-08-18T13:52:33.293895+0000","last_clean":"2024-08-18T13:52:33.293895+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:52:33.293895+0000","last_undegraded":"2024-08-18T13:52:33.293895+0000","last_fullsized":"2024-08-18T13:52:33.293895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'109","last_scrub_stamp":"2024-08-18T13:52:33.293641+0000","last_deep_scrub":"54'109","last_deep_scrub_stamp":"2024-08-18T13:52:33.293641+0000","last_clean_scrub_stamp":"2024-08-18T13:52:33.293641+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:59:10.451243+0000","scrub_duration":0.0039716550000000002,"objects_trimmed":0,"snaptrim_duration":0.0068339799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"54'105","reported_seq":163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.093354+0000","last_change":"2024-08-18T13:52:21.093276+0000","last_active":"2024-08-18T13:52:21.093354+0000","last_peered":"2024-08-18T13:52:21.093354+0000","last_clean":"2024-08-18T13:52:21.093354+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:52:21.093354+0000","last_undegraded":"2024-08-18T13:52:21.093354+0000","last_fullsized":"2024-08-18T13:52:21.093354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'105","last_scrub_stamp":"2024-08-18T13:52:21.093147+0000","last_deep_scrub":"54'105","last_deep_scrub_stamp":"2024-08-18T13:52:21.093147+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.093147+0000","objects_scrubbed":2,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:53:06.284734+0000","scrub_duration":0.015701764999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":103,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7766,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"54'135","reported_seq":201,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.254544+0000","last_change":"2024-08-18T13:52:04.254482+0000","last_active":"2024-08-18T13:52:04.254544+0000","last_peered":"2024-08-18T13:52:04.254544+0000","last_clean":"2024-08-18T13:52:04.254544+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:52:04.254544+0000","last_undegraded":"2024-08-18T13:52:04.254544+0000","last_fullsized":"2024-08-18T13:52:04.254544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'135","last_scrub_stamp":"2024-08-18T13:52:04.254271+0000","last_deep_scrub":"54'135","last_deep_scrub_stamp":"2024-08-18T13:52:04.254271+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.254271+0000","objects_scrubbed":5,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:13:13.411254+0000","scrub_duration":0.03253764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":128,"num_write_kb":12417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24866,"num_omap_keys":48,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"54'91","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.023472+0000","last_change":"2024-08-18T13:52:10.023405+0000","last_active":"2024-08-18T13:52:10.023472+0000","last_peered":"2024-08-18T13:52:10.023472+0000","last_clean":"2024-08-18T13:52:10.023472+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:52:10.023472+0000","last_undegraded":"2024-08-18T13:52:10.023472+0000","last_fullsized":"2024-08-18T13:52:10.023472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:10.023201+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:10.023201+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.023201+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:06.872974+0000","scrub_duration":0.003062024,"objects_trimmed":0,"snaptrim_duration":0.0053680339999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"54'664","reported_seq":3659,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:26.626366+0000","last_change":"2024-08-18T13:52:26.626281+0000","last_active":"2024-08-18T13:52:26.626366+0000","last_peered":"2024-08-18T13:52:26.626366+0000","last_clean":"2024-08-18T13:52:26.626366+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:52:26.626366+0000","last_undegraded":"2024-08-18T13:52:26.626366+0000","last_fullsized":"2024-08-18T13:52:26.626366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'664","last_scrub_stamp":"2024-08-18T13:52:26.626137+0000","last_deep_scrub":"54'664","last_deep_scrub_stamp":"2024-08-18T13:52:26.626137+0000","last_clean_scrub_stamp":"2024-08-18T13:52:26.626137+0000","objects_scrubbed":365,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:57.639383+0000","scrub_duration":0.14017384899999999,"objects_trimmed":0,"snaptrim_duration":0.0035122529999999999,"stat_sum":{"num_bytes":30328916,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":908,"num_read_kb":29115,"num_write":663,"num_write_kb":34071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"54'17","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.959979+0000","last_change":"2024-08-18T13:52:10.959908+0000","last_active":"2024-08-18T13:52:10.959979+0000","last_peered":"2024-08-18T13:52:10.959979+0000","last_clean":"2024-08-18T13:52:10.959979+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:52:10.959979+0000","last_undegraded":"2024-08-18T13:52:10.959979+0000","last_fullsized":"2024-08-18T13:52:10.959979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'17","last_scrub_stamp":"2024-08-18T13:52:10.959683+0000","last_deep_scrub":"54'17","last_deep_scrub_stamp":"2024-08-18T13:52:10.959683+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.959683+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:40.697579+0000","scrub_duration":0.013613793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":19,"num_write_kb":4249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21984,"num_omap_keys":42,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.601341+0000","last_change":"2024-08-18T13:52:07.601203+0000","last_active":"2024-08-18T13:52:07.601341+0000","last_peered":"2024-08-18T13:52:07.601341+0000","last_clean":"2024-08-18T13:52:07.601341+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:52:07.601341+0000","last_undegraded":"2024-08-18T13:52:07.601341+0000","last_fullsized":"2024-08-18T13:52:07.601341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:07.600983+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:07.600983+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.600983+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:43:21.683007+0000","scrub_duration":0.0035012659999999998,"objects_trimmed":0,"snaptrim_duration":0.0072935329999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"54'656","reported_seq":3235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:49.705587+0000","last_change":"2024-08-18T13:43:55.555171+0000","last_active":"2024-08-18T13:51:49.705587+0000","last_peered":"2024-08-18T13:51:49.705587+0000","last_clean":"2024-08-18T13:51:49.705587+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:51:49.705587+0000","last_undegraded":"2024-08-18T13:51:49.705587+0000","last_fullsized":"2024-08-18T13:51:49.705587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:38.406196+0000","last_clean_scrub_stamp":"2024-08-18T13:39:38.406196+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063112100000000003,"stat_sum":{"num_bytes":21201155,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":516,"num_read_kb":24278,"num_write":657,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"54'107","reported_seq":2181,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.370471+0000","last_change":"2024-08-18T13:52:21.370406+0000","last_active":"2024-08-18T13:52:21.370471+0000","last_peered":"2024-08-18T13:52:21.370471+0000","last_clean":"2024-08-18T13:52:21.370471+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:52:21.370471+0000","last_undegraded":"2024-08-18T13:52:21.370471+0000","last_fullsized":"2024-08-18T13:52:21.370471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'107","last_scrub_stamp":"2024-08-18T13:52:21.370198+0000","last_deep_scrub":"54'107","last_deep_scrub_stamp":"2024-08-18T13:52:21.370198+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.370198+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:44:33.767918+0000","scrub_duration":0.003909615,"objects_trimmed":0,"snaptrim_duration":0.0045422170000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"54'110","reported_seq":2184,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.497921+0000","last_change":"2024-08-18T13:52:21.497852+0000","last_active":"2024-08-18T13:52:21.497921+0000","last_peered":"2024-08-18T13:52:21.497921+0000","last_clean":"2024-08-18T13:52:21.497921+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:52:21.497921+0000","last_undegraded":"2024-08-18T13:52:21.497921+0000","last_fullsized":"2024-08-18T13:52:21.497921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'110","last_scrub_stamp":"2024-08-18T13:52:21.497653+0000","last_deep_scrub":"54'110","last_deep_scrub_stamp":"2024-08-18T13:52:21.497653+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.497653+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:28.676067+0000","scrub_duration":0.003899516,"objects_trimmed":0,"snaptrim_duration":0.0032950929999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"54'101","reported_seq":2185,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:17.440214+0000","last_change":"2024-08-18T13:52:17.440127+0000","last_active":"2024-08-18T13:52:17.440214+0000","last_peered":"2024-08-18T13:52:17.440214+0000","last_clean":"2024-08-18T13:52:17.440214+0000","last_became_active":"2024-08-18T13:40:16.610568+0000","last_became_peered":"2024-08-18T13:40:16.610568+0000","last_unstale":"2024-08-18T13:52:17.440214+0000","last_undegraded":"2024-08-18T13:52:17.440214+0000","last_fullsized":"2024-08-18T13:52:17.440214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'101","last_scrub_stamp":"2024-08-18T13:52:17.439868+0000","last_deep_scrub":"54'101","last_deep_scrub_stamp":"2024-08-18T13:52:17.439868+0000","last_clean_scrub_stamp":"2024-08-18T13:52:17.439868+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:06:03.535494+0000","scrub_duration":0.003706352,"objects_trimmed":0,"snaptrim_duration":0.0066503589999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"54'99","reported_seq":2174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.967255+0000","last_change":"2024-08-18T13:52:22.967191+0000","last_active":"2024-08-18T13:52:22.967255+0000","last_peered":"2024-08-18T13:52:22.967255+0000","last_clean":"2024-08-18T13:52:22.967255+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:52:22.967255+0000","last_undegraded":"2024-08-18T13:52:22.967255+0000","last_fullsized":"2024-08-18T13:52:22.967255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'99","last_scrub_stamp":"2024-08-18T13:52:22.966987+0000","last_deep_scrub":"54'99","last_deep_scrub_stamp":"2024-08-18T13:52:22.966987+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.966987+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:46:56.149355+0000","scrub_duration":0.0039138150000000002,"objects_trimmed":0,"snaptrim_duration":0.0056450420000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"54'80","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.895344+0000","last_change":"2024-08-18T13:52:05.895217+0000","last_active":"2024-08-18T13:52:05.895344+0000","last_peered":"2024-08-18T13:52:05.895344+0000","last_clean":"2024-08-18T13:52:05.895344+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:52:05.895344+0000","last_undegraded":"2024-08-18T13:52:05.895344+0000","last_fullsized":"2024-08-18T13:52:05.895344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'80","last_scrub_stamp":"2024-08-18T13:52:05.894985+0000","last_deep_scrub":"54'80","last_deep_scrub_stamp":"2024-08-18T13:52:05.894985+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.894985+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:51.990610+0000","scrub_duration":0.0040115990000000002,"objects_trimmed":0,"snaptrim_duration":0.0068042459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"54'74","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.251604+0000","last_change":"2024-08-18T13:52:04.251540+0000","last_active":"2024-08-18T13:52:04.251604+0000","last_peered":"2024-08-18T13:52:04.251604+0000","last_clean":"2024-08-18T13:52:04.251604+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:52:04.251604+0000","last_undegraded":"2024-08-18T13:52:04.251604+0000","last_fullsized":"2024-08-18T13:52:04.251604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'74","last_scrub_stamp":"2024-08-18T13:52:04.251343+0000","last_deep_scrub":"54'74","last_deep_scrub_stamp":"2024-08-18T13:52:04.251343+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.251343+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:35.264589+0000","scrub_duration":0.003478264,"objects_trimmed":0,"snaptrim_duration":0.0079157710000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"54'109","reported_seq":2183,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.271120+0000","last_change":"2024-08-18T13:52:29.271056+0000","last_active":"2024-08-18T13:52:29.271120+0000","last_peered":"2024-08-18T13:52:29.271120+0000","last_clean":"2024-08-18T13:52:29.271120+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:52:29.271120+0000","last_undegraded":"2024-08-18T13:52:29.271120+0000","last_fullsized":"2024-08-18T13:52:29.271120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'109","last_scrub_stamp":"2024-08-18T13:52:29.270852+0000","last_deep_scrub":"54'109","last_deep_scrub_stamp":"2024-08-18T13:52:29.270852+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.270852+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:16:03.188192+0000","scrub_duration":0.003274169,"objects_trimmed":0,"snaptrim_duration":0.0029546540000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"54'85","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.260651+0000","last_change":"2024-08-18T13:52:29.260514+0000","last_active":"2024-08-18T13:52:29.260651+0000","last_peered":"2024-08-18T13:52:29.260651+0000","last_clean":"2024-08-18T13:52:29.260651+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:52:29.260651+0000","last_undegraded":"2024-08-18T13:52:29.260651+0000","last_fullsized":"2024-08-18T13:52:29.260651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:29.260293+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:29.260293+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.260293+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:16:31.129273+0000","scrub_duration":0.0036944429999999999,"objects_trimmed":0,"snaptrim_duration":0.0055234810000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"54'92","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:28.509407+0000","last_change":"2024-08-18T13:52:28.509339+0000","last_active":"2024-08-18T13:52:28.509407+0000","last_peered":"2024-08-18T13:52:28.509407+0000","last_clean":"2024-08-18T13:52:28.509407+0000","last_became_active":"2024-08-18T13:40:16.614889+0000","last_became_peered":"2024-08-18T13:40:16.614889+0000","last_unstale":"2024-08-18T13:52:28.509407+0000","last_undegraded":"2024-08-18T13:52:28.509407+0000","last_fullsized":"2024-08-18T13:52:28.509407+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:28.509150+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:28.509150+0000","last_clean_scrub_stamp":"2024-08-18T13:52:28.509150+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:06:14.604775+0000","scrub_duration":0.0033533339999999999,"objects_trimmed":0,"snaptrim_duration":0.0033397829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:03.919727+0000","last_change":"2024-08-18T13:52:03.919668+0000","last_active":"2024-08-18T13:52:03.919727+0000","last_peered":"2024-08-18T13:52:03.919727+0000","last_clean":"2024-08-18T13:52:03.919727+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:52:03.919727+0000","last_undegraded":"2024-08-18T13:52:03.919727+0000","last_fullsized":"2024-08-18T13:52:03.919727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:03.919468+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:03.919468+0000","last_clean_scrub_stamp":"2024-08-18T13:52:03.919468+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:34.932714+0000","scrub_duration":0.0037821840000000001,"objects_trimmed":0,"snaptrim_duration":0.0070000389999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"54'95","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.465675+0000","last_change":"2024-08-18T13:52:04.465574+0000","last_active":"2024-08-18T13:52:04.465675+0000","last_peered":"2024-08-18T13:52:04.465675+0000","last_clean":"2024-08-18T13:52:04.465675+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:52:04.465675+0000","last_undegraded":"2024-08-18T13:52:04.465675+0000","last_fullsized":"2024-08-18T13:52:04.465675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'95","last_scrub_stamp":"2024-08-18T13:52:04.465382+0000","last_deep_scrub":"54'95","last_deep_scrub_stamp":"2024-08-18T13:52:04.465382+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.465382+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:27.412242+0000","scrub_duration":0.003183946,"objects_trimmed":0,"snaptrim_duration":0.0070451519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"54'88","reported_seq":2163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.037224+0000","last_change":"2024-08-18T13:52:30.037158+0000","last_active":"2024-08-18T13:52:30.037224+0000","last_peered":"2024-08-18T13:52:30.037224+0000","last_clean":"2024-08-18T13:52:30.037224+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:52:30.037224+0000","last_undegraded":"2024-08-18T13:52:30.037224+0000","last_fullsized":"2024-08-18T13:52:30.037224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:30.037003+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:30.037003+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.037003+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:04:20.479286+0000","scrub_duration":0.0027674930000000002,"objects_trimmed":0,"snaptrim_duration":0.0070254549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"54'98","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.332333+0000","last_change":"2024-08-18T13:52:22.332204+0000","last_active":"2024-08-18T13:52:22.332333+0000","last_peered":"2024-08-18T13:52:22.332333+0000","last_clean":"2024-08-18T13:52:22.332333+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:52:22.332333+0000","last_undegraded":"2024-08-18T13:52:22.332333+0000","last_fullsized":"2024-08-18T13:52:22.332333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'98","last_scrub_stamp":"2024-08-18T13:52:22.331993+0000","last_deep_scrub":"54'98","last_deep_scrub_stamp":"2024-08-18T13:52:22.331993+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.331993+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:49.278847+0000","scrub_duration":0.003320319,"objects_trimmed":0,"snaptrim_duration":0.003261548,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"54'85","reported_seq":2144,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.758780+0000","last_change":"2024-08-18T13:43:56.141939+0000","last_active":"2024-08-18T13:51:51.758780+0000","last_peered":"2024-08-18T13:51:51.758780+0000","last_clean":"2024-08-18T13:51:51.758780+0000","last_became_active":"2024-08-18T13:40:16.612760+0000","last_became_peered":"2024-08-18T13:40:16.612760+0000","last_unstale":"2024-08-18T13:51:51.758780+0000","last_undegraded":"2024-08-18T13:51:51.758780+0000","last_fullsized":"2024-08-18T13:51:51.758780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076223990000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.547065+0000","last_change":"2024-08-18T13:52:20.546999+0000","last_active":"2024-08-18T13:52:20.547065+0000","last_peered":"2024-08-18T13:52:20.547065+0000","last_clean":"2024-08-18T13:52:20.547065+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:52:20.547065+0000","last_undegraded":"2024-08-18T13:52:20.547065+0000","last_fullsized":"2024-08-18T13:52:20.547065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:20.546797+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:20.546797+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.546797+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:47:09.544407+0000","scrub_duration":0.0042093260000000002,"objects_trimmed":0,"snaptrim_duration":0.0027557649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"54'94","reported_seq":2169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.412704+0000","last_change":"2024-08-18T13:52:16.412631+0000","last_active":"2024-08-18T13:52:16.412704+0000","last_peered":"2024-08-18T13:52:16.412704+0000","last_clean":"2024-08-18T13:52:16.412704+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:52:16.412704+0000","last_undegraded":"2024-08-18T13:52:16.412704+0000","last_fullsized":"2024-08-18T13:52:16.412704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'94","last_scrub_stamp":"2024-08-18T13:52:16.412429+0000","last_deep_scrub":"54'94","last_deep_scrub_stamp":"2024-08-18T13:52:16.412429+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.412429+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:02.451289+0000","scrub_duration":0.003524293,"objects_trimmed":0,"snaptrim_duration":0.0069473399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"54'96","reported_seq":2171,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.437555+0000","last_change":"2024-08-18T13:52:22.437446+0000","last_active":"2024-08-18T13:52:22.437555+0000","last_peered":"2024-08-18T13:52:22.437555+0000","last_clean":"2024-08-18T13:52:22.437555+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:52:22.437555+0000","last_undegraded":"2024-08-18T13:52:22.437555+0000","last_fullsized":"2024-08-18T13:52:22.437555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'96","last_scrub_stamp":"2024-08-18T13:52:22.437239+0000","last_deep_scrub":"54'96","last_deep_scrub_stamp":"2024-08-18T13:52:22.437239+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.437239+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:52.175134+0000","scrub_duration":0.0037466019999999999,"objects_trimmed":0,"snaptrim_duration":0.0070196240000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:28.954154+0000","last_change":"2024-08-18T13:52:28.954074+0000","last_active":"2024-08-18T13:52:28.954154+0000","last_peered":"2024-08-18T13:52:28.954154+0000","last_clean":"2024-08-18T13:52:28.954154+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:52:28.954154+0000","last_undegraded":"2024-08-18T13:52:28.954154+0000","last_fullsized":"2024-08-18T13:52:28.954154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:28.953869+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:28.953869+0000","last_clean_scrub_stamp":"2024-08-18T13:52:28.953869+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:06.887322+0000","scrub_duration":0.0040004230000000003,"objects_trimmed":0,"snaptrim_duration":0.004509931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"54'92","reported_seq":2167,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.976679+0000","last_change":"2024-08-18T13:52:18.976615+0000","last_active":"2024-08-18T13:52:18.976679+0000","last_peered":"2024-08-18T13:52:18.976679+0000","last_clean":"2024-08-18T13:52:18.976679+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:52:18.976679+0000","last_undegraded":"2024-08-18T13:52:18.976679+0000","last_fullsized":"2024-08-18T13:52:18.976679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:18.976444+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:18.976444+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.976444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:27:33.305679+0000","scrub_duration":0.0033123919999999999,"objects_trimmed":0,"snaptrim_duration":0.0062343609999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"54'81","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:37.251183+0000","last_change":"2024-08-18T13:52:37.251112+0000","last_active":"2024-08-18T13:52:37.251183+0000","last_peered":"2024-08-18T13:52:37.251183+0000","last_clean":"2024-08-18T13:52:37.251183+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:52:37.251183+0000","last_undegraded":"2024-08-18T13:52:37.251183+0000","last_fullsized":"2024-08-18T13:52:37.251183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'81","last_scrub_stamp":"2024-08-18T13:52:37.250900+0000","last_deep_scrub":"54'81","last_deep_scrub_stamp":"2024-08-18T13:52:37.250900+0000","last_clean_scrub_stamp":"2024-08-18T13:52:37.250900+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:56:39.825671+0000","scrub_duration":0.0040894440000000002,"objects_trimmed":0,"snaptrim_duration":0.0072121939999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"54'77","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:17.643221+0000","last_change":"2024-08-18T13:52:17.643163+0000","last_active":"2024-08-18T13:52:17.643221+0000","last_peered":"2024-08-18T13:52:17.643221+0000","last_clean":"2024-08-18T13:52:17.643221+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:52:17.643221+0000","last_undegraded":"2024-08-18T13:52:17.643221+0000","last_fullsized":"2024-08-18T13:52:17.643221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'77","last_scrub_stamp":"2024-08-18T13:52:17.642974+0000","last_deep_scrub":"54'77","last_deep_scrub_stamp":"2024-08-18T13:52:17.642974+0000","last_clean_scrub_stamp":"2024-08-18T13:52:17.642974+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:17.078190+0000","scrub_duration":0.0036672470000000002,"objects_trimmed":0,"snaptrim_duration":0.0057680220000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"54'90","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:25.323806+0000","last_change":"2024-08-18T13:52:25.323732+0000","last_active":"2024-08-18T13:52:25.323806+0000","last_peered":"2024-08-18T13:52:25.323806+0000","last_clean":"2024-08-18T13:52:25.323806+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:52:25.323806+0000","last_undegraded":"2024-08-18T13:52:25.323806+0000","last_fullsized":"2024-08-18T13:52:25.323806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:25.323493+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:25.323493+0000","last_clean_scrub_stamp":"2024-08-18T13:52:25.323493+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:19:16.878783+0000","scrub_duration":0.0036337510000000002,"objects_trimmed":0,"snaptrim_duration":0.0049025989999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"54'98","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.239723+0000","last_change":"2024-08-18T13:52:30.239659+0000","last_active":"2024-08-18T13:52:30.239723+0000","last_peered":"2024-08-18T13:52:30.239723+0000","last_clean":"2024-08-18T13:52:30.239723+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:52:30.239723+0000","last_undegraded":"2024-08-18T13:52:30.239723+0000","last_fullsized":"2024-08-18T13:52:30.239723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'98","last_scrub_stamp":"2024-08-18T13:52:30.239450+0000","last_deep_scrub":"54'98","last_deep_scrub_stamp":"2024-08-18T13:52:30.239450+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.239450+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:32:05.377873+0000","scrub_duration":0.0037021020000000001,"objects_trimmed":0,"snaptrim_duration":0.0029569129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"54'83","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.837796+0000","last_change":"2024-08-18T13:52:34.837716+0000","last_active":"2024-08-18T13:52:34.837796+0000","last_peered":"2024-08-18T13:52:34.837796+0000","last_clean":"2024-08-18T13:52:34.837796+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:52:34.837796+0000","last_undegraded":"2024-08-18T13:52:34.837796+0000","last_fullsized":"2024-08-18T13:52:34.837796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'83","last_scrub_stamp":"2024-08-18T13:52:34.837509+0000","last_deep_scrub":"54'83","last_deep_scrub_stamp":"2024-08-18T13:52:34.837509+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.837509+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:49.624348+0000","scrub_duration":0.0037147289999999999,"objects_trimmed":0,"snaptrim_duration":0.0062586680000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"54'74","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.923219+0000","last_change":"2024-08-18T13:52:04.923145+0000","last_active":"2024-08-18T13:52:04.923219+0000","last_peered":"2024-08-18T13:52:04.923219+0000","last_clean":"2024-08-18T13:52:04.923219+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:52:04.923219+0000","last_undegraded":"2024-08-18T13:52:04.923219+0000","last_fullsized":"2024-08-18T13:52:04.923219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'74","last_scrub_stamp":"2024-08-18T13:52:04.922933+0000","last_deep_scrub":"54'74","last_deep_scrub_stamp":"2024-08-18T13:52:04.922933+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.922933+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:50.961793+0000","scrub_duration":0.0036959599999999999,"objects_trimmed":0,"snaptrim_duration":0.0064056340000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"54'105","reported_seq":2179,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.843758+0000","last_change":"2024-08-18T13:52:31.843695+0000","last_active":"2024-08-18T13:52:31.843758+0000","last_peered":"2024-08-18T13:52:31.843758+0000","last_clean":"2024-08-18T13:52:31.843758+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:52:31.843758+0000","last_undegraded":"2024-08-18T13:52:31.843758+0000","last_fullsized":"2024-08-18T13:52:31.843758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'105","last_scrub_stamp":"2024-08-18T13:52:31.843506+0000","last_deep_scrub":"54'105","last_deep_scrub_stamp":"2024-08-18T13:52:31.843506+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.843506+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:54:05.314296+0000","scrub_duration":0.003779241,"objects_trimmed":0,"snaptrim_duration":0.0050911749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"54'93","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:25.511140+0000","last_change":"2024-08-18T13:52:25.511071+0000","last_active":"2024-08-18T13:52:25.511140+0000","last_peered":"2024-08-18T13:52:25.511140+0000","last_clean":"2024-08-18T13:52:25.511140+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:52:25.511140+0000","last_undegraded":"2024-08-18T13:52:25.511140+0000","last_fullsized":"2024-08-18T13:52:25.511140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'93","last_scrub_stamp":"2024-08-18T13:52:25.510879+0000","last_deep_scrub":"54'93","last_deep_scrub_stamp":"2024-08-18T13:52:25.510879+0000","last_clean_scrub_stamp":"2024-08-18T13:52:25.510879+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:50.842504+0000","scrub_duration":0.0038326670000000001,"objects_trimmed":0,"snaptrim_duration":0.0068654359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"54'102","reported_seq":2176,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.859948+0000","last_change":"2024-08-18T13:52:29.859863+0000","last_active":"2024-08-18T13:52:29.859948+0000","last_peered":"2024-08-18T13:52:29.859948+0000","last_clean":"2024-08-18T13:52:29.859948+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:52:29.859948+0000","last_undegraded":"2024-08-18T13:52:29.859948+0000","last_fullsized":"2024-08-18T13:52:29.859948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'102","last_scrub_stamp":"2024-08-18T13:52:29.859674+0000","last_deep_scrub":"54'102","last_deep_scrub_stamp":"2024-08-18T13:52:29.859674+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.859674+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:58:17.203374+0000","scrub_duration":0.0033213359999999998,"objects_trimmed":0,"snaptrim_duration":0.0055002549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"54'76","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.564884+0000","last_change":"2024-08-18T13:52:29.564814+0000","last_active":"2024-08-18T13:52:29.564884+0000","last_peered":"2024-08-18T13:52:29.564884+0000","last_clean":"2024-08-18T13:52:29.564884+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:52:29.564884+0000","last_undegraded":"2024-08-18T13:52:29.564884+0000","last_fullsized":"2024-08-18T13:52:29.564884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'76","last_scrub_stamp":"2024-08-18T13:52:29.564588+0000","last_deep_scrub":"54'76","last_deep_scrub_stamp":"2024-08-18T13:52:29.564588+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.564588+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:32.503951+0000","scrub_duration":0.0037994510000000001,"objects_trimmed":0,"snaptrim_duration":0.0069338170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.940841+0000","last_change":"2024-08-18T13:52:20.940751+0000","last_active":"2024-08-18T13:52:20.940841+0000","last_peered":"2024-08-18T13:52:20.940841+0000","last_clean":"2024-08-18T13:52:20.940841+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:52:20.940841+0000","last_undegraded":"2024-08-18T13:52:20.940841+0000","last_fullsized":"2024-08-18T13:52:20.940841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:20.940535+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:20.940535+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.940535+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:23.879899+0000","scrub_duration":0.003381294,"objects_trimmed":0,"snaptrim_duration":0.0044622389999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"54'76","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.995177+0000","last_change":"2024-08-18T13:52:27.995104+0000","last_active":"2024-08-18T13:52:27.995177+0000","last_peered":"2024-08-18T13:52:27.995177+0000","last_clean":"2024-08-18T13:52:27.995177+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:52:27.995177+0000","last_undegraded":"2024-08-18T13:52:27.995177+0000","last_fullsized":"2024-08-18T13:52:27.995177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'76","last_scrub_stamp":"2024-08-18T13:52:27.994891+0000","last_deep_scrub":"54'76","last_deep_scrub_stamp":"2024-08-18T13:52:27.994891+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.994891+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:36.746461+0000","scrub_duration":0.0032603279999999998,"objects_trimmed":0,"snaptrim_duration":0.0035965799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"54'86","reported_seq":2150,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:51:51.778898+0000","last_change":"2024-08-18T13:43:55.552484+0000","last_active":"2024-08-18T13:51:51.778898+0000","last_peered":"2024-08-18T13:51:51.778898+0000","last_clean":"2024-08-18T13:51:51.778898+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:51:51.778898+0000","last_undegraded":"2024-08-18T13:51:51.778898+0000","last_fullsized":"2024-08-18T13:51:51.778898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T13:39:34.358931+0000","last_clean_scrub_stamp":"2024-08-18T13:39:34.358931+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067690759999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":938443088,"num_objects":10906,"num_object_clones":0,"num_object_copies":43624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10906,"num_whiteouts":0,"num_read":21895,"num_read_kb":705026,"num_write":21484,"num_write_kb":1061752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2001649664,"data_stored":2001649664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5769,"ondisk_log_size":5769,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":184848248,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":586,"num_read_kb":685208,"num_write":6435,"num_write_kb":186300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576095,"num_omap_keys":1100,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":369762304,"data_stored":369696496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6483,"ondisk_log_size":6483,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600108,"kb_used_data":294436,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93771732,"statfs":{"total":96636764160,"available":96022253568,"internally_reserved":0,"allocated":301502464,"data_stored":301200284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2999999999999998,"5min":1.026,"15min":0.878},"min":{"1min":0.48299999999999998,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":25.744,"5min":25.744,"15min":25.744},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":0.84999999999999998,"15min":0.82599999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.33100000000000002},"max":{"1min":8.1679999999999993,"5min":8.1679999999999993,"15min":8.1679999999999993},"last":1.234}]},{"osd":1,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":0.90700000000000003,"15min":0.98599999999999999},"min":{"1min":0.5,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":10.151999999999999,"5min":10.151999999999999,"15min":39.203000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":2.323,"5min":1.111,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":25.852,"5min":25.852,"15min":25.852},"last":0.90300000000000002}]},{"osd":2,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.3100000000000001,"5min":1.121,"15min":0.95099999999999996},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.39400000000000002},"max":{"1min":25.818999999999999,"5min":25.818999999999999,"15min":25.818999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.85199999999999998},"min":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.35799999999999998},"max":{"1min":1.133,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.069}]},{"osd":3,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.0840000000000001,"15min":1.135},"min":{"1min":0.64700000000000002,"5min":0.42699999999999999,"15min":0.377},"max":{"1min":17.300999999999998,"5min":17.300999999999998,"15min":42.905000000000001},"last":1.004},{"interface":"front","average":{"1min":2.4689999999999999,"5min":1.238,"15min":1.115},"min":{"1min":0.75600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":25.786999999999999,"5min":25.786999999999999,"15min":25.786999999999999},"last":0.97899999999999998}]},{"osd":4,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.97199999999999998,"15min":0.94499999999999995},"min":{"1min":0.58499999999999996,"5min":0.376,"15min":0.29699999999999999},"max":{"1min":8.3870000000000005,"5min":8.3870000000000005,"15min":8.3870000000000005},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.171,"15min":1.23},"min":{"1min":0.79800000000000004,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":17.056999999999999,"5min":17.056999999999999,"15min":52.673000000000002},"last":0.84299999999999997}]},{"osd":5,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4329999999999998,"5min":1.274,"15min":1.119},"min":{"1min":0.44,"5min":0.44,"15min":0.33700000000000002},"max":{"1min":25.881,"5min":25.881,"15min":25.881},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.1080000000000001,"15min":1.073},"min":{"1min":0.73899999999999999,"5min":0.54200000000000004,"15min":0.37},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.1839999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.1659999999999999,"15min":1.1259999999999999},"min":{"1min":0.83799999999999997,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":8.0510000000000002,"5min":8.0510000000000002,"15min":8.0510000000000002},"last":1.127},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.1990000000000001,"15min":1.26},"min":{"1min":0.85499999999999998,"5min":0.49399999999999999,"15min":0.435},"max":{"1min":10.523,"5min":10.523,"15min":39.161000000000001},"last":1.097}]}]},{"osd":6,"up_from":15,"seq":64424509605,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610072,"kb_used_data":312720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93761768,"statfs":{"total":96636764160,"available":96012050432,"internally_reserved":0,"allocated":320225280,"data_stored":319926507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.76500000000000001,"15min":0.88},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.165,"5min":1.21,"15min":4.532},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77600000000000002,"15min":0.83499999999999996},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.248},"max":{"1min":1.258,"5min":1.3200000000000001,"15min":1.9650000000000001},"last":0.92200000000000004}]},{"osd":1,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.92000000000000004},"min":{"1min":0.42399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2749999999999999,"5min":1.294,"15min":4.468},"last":1.337},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.91900000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.30499999999999999},"max":{"1min":1.1990000000000001,"5min":1.286,"15min":2.3620000000000001},"last":1.2190000000000001}]},{"osd":2,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77700000000000002,"15min":0.83899999999999997},"min":{"1min":0.49299999999999999,"5min":0.40200000000000002,"15min":0.28000000000000003},"max":{"1min":1.085,"5min":1.177,"15min":1.413},"last":1.022},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.97299999999999998},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1060000000000001,"5min":1.468,"15min":4.5},"last":1.181}]},{"osd":3,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88700000000000001,"15min":0.94199999999999995},"min":{"1min":0.60899999999999999,"5min":0.41999999999999998,"15min":0.35399999999999998},"max":{"1min":1.1919999999999999,"5min":1.276,"15min":4.3250000000000002},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.875,"15min":0.91600000000000004},"min":{"1min":0.55400000000000005,"5min":0.47799999999999998,"15min":0.32900000000000001},"max":{"1min":1.0920000000000001,"5min":1.306,"15min":1.8069999999999999},"last":0.96299999999999997}]},{"osd":4,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.92700000000000005,"15min":1.0109999999999999},"min":{"1min":0.47699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.228,"5min":1.552,"15min":5.2549999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":1.248,"5min":1.7230000000000001,"15min":4.6600000000000001},"last":1.0489999999999999}]},{"osd":5,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91300000000000003,"15min":1.0129999999999999},"min":{"1min":0.54900000000000004,"5min":0.45400000000000001,"15min":0.373},"max":{"1min":1.179,"5min":1.3160000000000001,"15min":4.4009999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.042,"15min":1.1499999999999999},"min":{"1min":0.505,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.272,"5min":1.6879999999999999,"15min":6.1260000000000003},"last":1.4079999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:49 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0669999999999999,"15min":1.111},"min":{"1min":0.67600000000000005,"5min":0.55500000000000005,"15min":0.40500000000000003},"max":{"1min":1.3009999999999999,"5min":1.4830000000000001,"15min":1.9299999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0609999999999999,"15min":1.1080000000000001},"min":{"1min":0.67500000000000004,"5min":0.46300000000000002,"15min":0.40200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":2.1150000000000002},"last":1.125}]}]},{"osd":4,"up_from":15,"seq":64424509603,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570388,"kb_used_data":273036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93801452,"statfs":{"total":96636764160,"available":96052686848,"internally_reserved":0,"allocated":279588864,"data_stored":279281600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.24299999999999999},"max":{"1min":1.1679999999999999,"5min":1.204,"15min":1.22},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":1.085,"5min":1.339,"15min":1.73},"last":0.70299999999999996}]},{"osd":1,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.82499999999999996,"15min":0.86799999999999999},"min":{"1min":0.54100000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.2909999999999999,"5min":1.415,"15min":3.6299999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.55500000000000005,"5min":0.36699999999999999,"15min":0.27600000000000002},"max":{"1min":1.0589999999999999,"5min":1.28,"15min":1.5169999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83099999999999996,"15min":0.85799999999999998},"min":{"1min":0.51000000000000001,"5min":0.45600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.58299999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":3.5920000000000001},"last":1.1220000000000001}]},{"osd":3,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92100000000000004,"15min":0.91800000000000004},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.309},"max":{"1min":1.111,"5min":1.387,"15min":1.45},"last":0.995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.641,"15min":1.641},"last":1.0800000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.94099999999999995,"15min":0.97299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.313},"max":{"1min":1.3640000000000001,"5min":1.575,"15min":3.7789999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.998,"15min":1.0149999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3129999999999999,"5min":1.754,"15min":1.754},"last":1.147}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0580000000000001,"15min":1.079},"min":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.504},"max":{"1min":1.6259999999999999,"5min":1.901,"15min":7.8490000000000002},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.016,"15min":1.0629999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2809999999999999,"5min":1.5229999999999999,"15min":6.3380000000000001},"last":0.752}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.40000000000000002},"max":{"1min":1.2470000000000001,"5min":1.7470000000000001,"15min":1.8400000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.149,"5min":1.151,"15min":1.1819999999999999},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.42399999999999999},"max":{"1min":1.6850000000000001,"5min":2.4569999999999999,"15min":3.7410000000000001},"last":1.0549999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542308,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683436,"kb_used_data":373604,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93688404,"statfs":{"total":96636764160,"available":95936925696,"internally_reserved":0,"allocated":382570496,"data_stored":382269906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.29499999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.89500000000000002,"5min":1.2050000000000001,"15min":1.2170000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67800000000000005,"15min":0.69299999999999995},"min":{"1min":0.44900000000000001,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":0.90400000000000003,"5min":1.413,"15min":1.413},"last":0.89200000000000002}]},{"osd":1,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.68600000000000005},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86499999999999999,"5min":1.46,"15min":3.4820000000000002},"last":0.442},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.751},"min":{"1min":0.47599999999999998,"5min":0.38,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.2669999999999999,"15min":1.8979999999999999},"last":0.67600000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79200000000000004,"15min":0.81499999999999995},"min":{"1min":0.48599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1910000000000001,"5min":1.6419999999999999,"15min":2.415},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.87,"15min":0.86899999999999999},"min":{"1min":0.57799999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.2589999999999999,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.76500000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.173,"5min":1.681,"15min":3.2490000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.052},"min":{"1min":0.66300000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5189999999999999,"5min":2.4870000000000001,"15min":3.8799999999999999},"last":0.91600000000000004}]},{"osd":5,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.996,"15min":4.1059999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0089999999999999,"15min":1.05},"min":{"1min":0.63200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1719999999999999,"5min":1.627,"15min":3.2799999999999998},"last":0.72899999999999998}]},{"osd":6,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.504,"5min":0.504,"15min":0.41999999999999998},"max":{"1min":1.3480000000000001,"5min":1.833,"15min":2.1080000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.054,"15min":1.0880000000000001},"min":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.246,"5min":1.573,"15min":2.7469999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Aug 18 13:51:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4419999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.087,"15min":1.0960000000000001},"min":{"1min":0.71499999999999997,"5min":0.63200000000000001,"15min":0.51100000000000001},"max":{"1min":1.583,"5min":1.9039999999999999,"15min":2.1429999999999998},"last":0.748}]}]},{"osd":2,"up_from":14,"seq":60129542307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530780,"kb_used_data":229268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93841060,"statfs":{"total":96636764160,"available":96093245440,"internally_reserved":0,"allocated":234770432,"data_stored":234477374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.32700000000000001,"5min":0.24099999999999999,"15min":0.186},"max":{"1min":1.0369999999999999,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":1.0960000000000001,"5min":3.27,"15min":3.27},"last":0.626}]},{"osd":1,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.375,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.038,"5min":1.1040000000000001,"15min":1.696},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":3.2290000000000001,"15min":3.2290000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.78200000000000003},"min":{"1min":0.39600000000000002,"5min":0.28899999999999998,"15min":0.253},"max":{"1min":0.995,"5min":1.077,"15min":12.468999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.84399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35899999999999999,"15min":0.27500000000000002},"max":{"1min":1.077,"5min":1.3540000000000001,"15min":1.744},"last":1.042}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.49299999999999999,"15min":0.40100000000000002},"max":{"1min":1.2709999999999999,"5min":1.681,"15min":1.8480000000000001},"last":1.208},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.94799999999999995},"min":{"1min":0.46899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2609999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":1.117}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.95699999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1759999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39000000000000001},"max":{"1min":1.149,"5min":1.498,"15min":1.627},"last":0.99399999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96899999999999997,"15min":0.98799999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.42199999999999999},"max":{"1min":1.22,"5min":1.4570000000000001,"15min":1.649},"last":1.153},{"interface":"front","average":{"1min":1.012,"5min":1.0309999999999999,"15min":1.0649999999999999},"min":{"1min":0.67000000000000004,"5min":0.51900000000000002,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":1.5820000000000001,"15min":1.905},"last":1.2869999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0549999999999999,"15min":1.0880000000000001},"min":{"1min":0.65400000000000003,"5min":0.60399999999999998,"15min":0.436},"max":{"1min":1.3140000000000001,"5min":1.5089999999999999,"15min":1.9490000000000001},"last":1.335},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.0149999999999999},"min":{"1min":0.57899999999999996,"5min":0.53200000000000003,"15min":0.46400000000000002},"max":{"1min":1.343,"5min":1.4379999999999999,"15min":1.7669999999999999},"last":1.2450000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542307,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677976,"kb_used_data":376464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93693864,"statfs":{"total":96636764160,"available":95942516736,"internally_reserved":0,"allocated":385499136,"data_stored":385199863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.65200000000000002},"min":{"1min":0.28899999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.94899999999999995,"5min":1.3009999999999999,"15min":3.1160000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.72099999999999997},"min":{"1min":0.30599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.107,"5min":1.327,"15min":3.718},"last":0.79100000000000004}]},{"osd":2,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.0549999999999999,"5min":4.968,"15min":4.968},"last":0.749},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.754,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33400000000000002,"15min":0.20899999999999999},"max":{"1min":1.327,"5min":1.3560000000000001,"15min":9.3309999999999995},"last":0.70499999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78100000000000003,"15min":0.82099999999999995},"min":{"1min":0.39500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2569999999999999,"5min":1.4810000000000001,"15min":1.927},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79400000000000004,"15min":0.84899999999999998},"min":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":1.46,"5min":1.46,"15min":6.8559999999999999},"last":1.018}]},{"osd":4,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.28000000000000003},"max":{"1min":1.488,"5min":5.1020000000000003,"15min":5.6529999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.63400000000000001,"5min":0.39700000000000002,"15min":0.28599999999999998},"max":{"1min":1.4410000000000001,"5min":1.554,"15min":1.5740000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3859999999999999,"5min":1.546,"15min":3.7570000000000001},"last":1.155},{"interface":"front","average":{"1min":1,"5min":1.04,"15min":1.1040000000000001},"min":{"1min":0.47799999999999998,"5min":0.46400000000000002,"15min":0.45700000000000002},"max":{"1min":1.591,"5min":5.1799999999999997,"15min":19.027999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.40699999999999997},"max":{"1min":1.5529999999999999,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":1.375},{"interface":"front","average":{"1min":1.113,"5min":1.0880000000000001,"15min":1.131},"min":{"1min":0.54900000000000004,"5min":0.49099999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.621,"15min":1.861},"last":1.228}]},{"osd":7,"last update":"Sun Aug 18 13:51:41 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.069,"15min":1.119},"min":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.58799999999999997},"max":{"1min":1.383,"5min":1.526,"15min":3.8100000000000001},"last":1.216},{"interface":"front","average":{"1min":1.083,"5min":1.1120000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40000000000000002},"max":{"1min":1.431,"5min":5.3099999999999996,"15min":5.3099999999999996},"last":1.2849999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542307,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549780,"kb_used_data":248268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93822060,"statfs":{"total":96636764160,"available":96073789440,"internally_reserved":0,"allocated":254226432,"data_stored":253919977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61399999999999999,"15min":0.65400000000000003},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":5.3159999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.33500000000000002,"5min":0.254,"15min":0.19500000000000001},"max":{"1min":0.93600000000000005,"5min":1.2370000000000001,"15min":3.4809999999999999},"last":1.004}]},{"osd":2,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.47099999999999997,"5min":0.32600000000000001,"15min":0.26300000000000001},"max":{"1min":0.99199999999999999,"5min":1.222,"15min":3.3029999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.67400000000000004},"min":{"1min":0.373,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.28},"last":0.95899999999999996}]},{"osd":3,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.82399999999999995},"min":{"1min":0.45800000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.288,"5min":1.288,"15min":5.399},"last":0.876},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.86799999999999999,"15min":0.93100000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.33900000000000002},"max":{"1min":1.1799999999999999,"5min":1.365,"15min":5.0359999999999996},"last":1.0820000000000001}]},{"osd":4,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.90200000000000002,"15min":0.92000000000000004},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.37},"max":{"1min":1.3899999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.278},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":8.7799999999999994},"last":1.048}]},{"osd":5,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96099999999999997,"15min":0.97499999999999998},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.35499999999999998},"max":{"1min":1.4770000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.202},{"interface":"front","average":{"1min":1.087,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.52900000000000003,"5min":0.43099999999999999,"15min":0.40300000000000002},"max":{"1min":1.51,"5min":1.52,"15min":1.8879999999999999},"last":1.339}]},{"osd":6,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.77800000000000002,"5min":0.46500000000000002,"15min":0.374},"max":{"1min":1.423,"5min":1.599,"15min":3.7200000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.042,"15min":1.085},"min":{"1min":0.68600000000000005,"5min":0.58099999999999996,"15min":0.52600000000000002},"max":{"1min":1.55,"5min":1.659,"15min":3.2789999999999999},"last":1.377}]},{"osd":7,"last update":"Sun Aug 18 13:51:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.143,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.60599999999999998,"15min":0.40300000000000002},"max":{"1min":1.514,"5min":1.704,"15min":36.295000000000002},"last":1.319},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.171,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.55300000000000005,"15min":0.42999999999999999},"max":{"1min":1.5740000000000001,"5min":1.706,"15min":3.448},"last":1.4830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515540,"kb_used_data":214028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93856300,"statfs":{"total":96636764160,"available":96108851200,"internally_reserved":0,"allocated":219164672,"data_stored":218869425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.76000000000000001,"15min":0.80100000000000005},"min":{"1min":0.36399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.153,"5min":1.204,"15min":1.901},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81699999999999995,"15min":0.84999999999999998},"min":{"1min":0.378,"5min":0.376,"15min":0.376},"max":{"1min":1.2410000000000001,"5min":1.2509999999999999,"15min":4.0810000000000004},"last":0.87}]},{"osd":1,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80100000000000005,"15min":0.81999999999999995},"min":{"1min":0.59899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1950000000000001,"5min":1.429,"15min":2.4809999999999999},"last":1.093},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.88800000000000001,"15min":0.90600000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.456,"15min":1.7749999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.91700000000000004,"15min":0.92800000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.41799999999999998},"max":{"1min":1.196,"5min":1.524,"15min":1.774},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.92600000000000005},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.32500000000000001},"max":{"1min":1.2609999999999999,"5min":1.3160000000000001,"15min":4.1559999999999997},"last":0.98699999999999999}]},{"osd":3,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":0.97199999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.36199999999999999},"max":{"1min":1.1719999999999999,"5min":1.4410000000000001,"15min":4.5510000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.97599999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.52400000000000002},"max":{"1min":1.2350000000000001,"5min":1.4470000000000001,"15min":3.976},"last":1.018}]},{"osd":4,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.97099999999999997,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.7909999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0660000000000001,"15min":1.1120000000000001},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.47899999999999998},"max":{"1min":1.2669999999999999,"5min":1.6839999999999999,"15min":2.532},"last":1.0529999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.03,"15min":1.085},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3109999999999999,"5min":1.532,"15min":4.2050000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0569999999999999,"15min":1.113},"min":{"1min":0.56699999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.5149999999999999,"5min":1.6479999999999999,"15min":2.911},"last":0.77900000000000003}]},{"osd":7,"last update":"Sun Aug 18 13:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.089,"15min":1.165},"min":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.2989999999999999,"5min":1.659,"15min":2.569},"last":1.147},{"interface":"front","average":{"1min":1.119,"5min":1.1359999999999999,"15min":1.361},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.40400000000000003},"max":{"1min":1.407,"5min":1.605,"15min":58.290999999999997},"last":1.119}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41955328,"data_stored":41943666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50331808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21549056,"data_stored":21547239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34140160,"data_stored":34130811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":54263808,"data_stored":54247785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41947136,"data_stored":41943130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50332308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":75227136,"data_stored":75219749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211058688,"data_stored":211058688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334540800,"data_stored":334540800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212602880,"data_stored":212602880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347811840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":224706560,"data_stored":224706560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176599040,"data_stored":176599040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":269266944,"data_stored":269266944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":225062912,"data_stored":225062912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:52:44.744 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.745 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.745 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.745 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.745 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.746 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.746 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.747 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.747 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.747 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.747 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.747 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-08-18T13:39:38.406196+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=38, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.748 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.748 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-08-18T13:39:34.358931+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=39, tm_sec=34, tm_wday=6, tm_yday=231, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=18, tm_hour=13, tm_min=51, tm_sec=58, tm_wday=6, tm_yday=231, tm_isdst=0) 2024-08-18T13:52:44.748 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-18T13:53:04.749 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T13:53:04.868 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.865+0000 7f0df0837700 1 Processor -- start 2024-08-18T13:53:04.869 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.865+0000 7f0df0837700 1 -- start start 2024-08-18T13:53:04.871 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0dec0be5a0 0x7f0dec0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:04.871 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec06de00 con 0x7f0dec06e4c0 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec0c2da0 con 0x7f0dec0c22a0 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec06fac0 con 0x7f0dec0be5a0 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dead9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0dec0be5a0 0x7f0dec0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0deb59e700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f0dec06e4c0 0x7f0dec151cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53904/0 (socket says 172.21.15.118:53904) 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0deb59e700 1 -- 172.21.15.118:0/1333522193 learned_addr learned my addr 172.21.15.118:0/1333522193 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dead9d700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0dec0be5a0 0x7f0dec0c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41264/0 (socket says 172.21.15.118:41264) 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dea59c700 1 -- 172.21.15.118:0/1333522193 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094572100 0 0) 0x7f0dec0c2da0 con 0x7f0dec0c22a0 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dea59c700 1 -- 172.21.15.118:0/1333522193 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd8003570 con 0x7f0dec0c22a0 2024-08-18T13:53:04.872 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dead9d700 1 -- 172.21.15.118:0/1333522193 >> v1:172.21.15.178:6789/0 conn(0x7f0dec0c22a0 legacy=0x7f0dec14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:04.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dead9d700 1 -- 172.21.15.118:0/1333522193 >> v1:172.21.15.118:6789/0 conn(0x7f0dec06e4c0 legacy=0x7f0dec151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:04.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dea59c700 1 -- 172.21.15.118:0/1333522193 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464146142 0 0) 0x7f0dec06de00 con 0x7f0dec06e4c0 2024-08-18T13:53:04.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dead9d700 1 -- 172.21.15.118:0/1333522193 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0c3080 con 0x7f0dec0be5a0 2024-08-18T13:53:04.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dead9d700 1 --2- 172.21.15.118:0/1333522193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec0be5a0 0x7f0dec0c7fc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098e0 tx=0x7f0ddc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5fa69b25ea2765 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:04.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dea59c700 1 -- 172.21.15.118:0/1333522193 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00b640 con 0x7f0dec0be5a0 2024-08-18T13:53:04.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dea59c700 1 -- 172.21.15.118:0/1333522193 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc00b7a0 con 0x7f0dec0be5a0 2024-08-18T13:53:04.873 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0dea59c700 1 -- 172.21.15.118:0/1333522193 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00bb40 con 0x7f0dec0be5a0 2024-08-18T13:53:04.874 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- 172.21.15.118:0/1333522193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec0be5a0 msgr2=0x7f0dec0c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:04.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 --2- 172.21.15.118:0/1333522193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec0be5a0 0x7f0dec0c7fc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098e0 tx=0x7f0ddc0091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:04.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- 172.21.15.118:0/1333522193 shutdown_connections 2024-08-18T13:53:04.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 --2- 172.21.15.118:0/1333522193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec0be5a0 0x7f0dec0c7fc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:04.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- 172.21.15.118:0/1333522193 >> 172.21.15.118:0/1333522193 conn(0x7f0dec06c020 msgr2=0x7f0dec0c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:04.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- 172.21.15.118:0/1333522193 shutdown_connections 2024-08-18T13:53:04.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.869+0000 7f0df0837700 1 -- 172.21.15.118:0/1333522193 wait complete. 2024-08-18T13:53:04.875 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 Processor -- start 2024-08-18T13:53:04.876 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 -- start start 2024-08-18T13:53:04.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec06e4c0 0x7f0dec14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:04.878 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0dec0be5a0 0x7f0dec14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dec0c22a0 0x7f0dec138af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec14eb40 con 0x7f0dec06e4c0 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec13cd10 con 0x7f0dec0c22a0 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0df0837700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec14d6f0 con 0x7f0dec0be5a0 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec06e4c0 0x7f0dec14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec06e4c0 0x7f0dec14a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41274/0 (socket says 172.21.15.118:41274) 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0deb59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dec0c22a0 0x7f0dec138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:04.879 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 -- 172.21.15.118:0/2161989972 learned_addr learned my addr 172.21.15.118:0/2161989972 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:04.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0de3fff700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0dec0be5a0 0x7f0dec14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:04.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 -- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0dec0be5a0 msgr2=0x7f0dec14a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:04.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0dec0be5a0 0x7f0dec14a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:04.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 -- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dec0c22a0 msgr2=0x7f0dec138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:04.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dec0c22a0 0x7f0dec138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:04.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0dead9d700 1 -- 172.21.15.118:0/2161989972 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc009420 con 0x7f0dec06e4c0 2024-08-18T13:53:04.880 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.873+0000 7f0deb59e700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dec0c22a0 0x7f0dec138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:53:04.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0dead9d700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec06e4c0 0x7f0dec14a4b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009f80 tx=0x7f0ddc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:04.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0de2ffd700 1 -- 172.21.15.118:0/2161989972 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc01f070 con 0x7f0dec06e4c0 2024-08-18T13:53:04.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0de2ffd700 1 -- 172.21.15.118:0/2161989972 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc002f70 con 0x7f0dec06e4c0 2024-08-18T13:53:04.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec067e30 con 0x7f0dec06e4c0 2024-08-18T13:53:04.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0de2ffd700 1 -- 172.21.15.118:0/2161989972 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00b910 con 0x7f0dec06e4c0 2024-08-18T13:53:04.881 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec13c750 con 0x7f0dec06e4c0 2024-08-18T13:53:04.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec138810 con 0x7f0dec06e4c0 2024-08-18T13:53:04.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0de2ffd700 1 -- 172.21.15.118:0/2161989972 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0ddc023070 con 0x7f0dec06e4c0 2024-08-18T13:53:04.882 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0de2ffd700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dcc060890 0x7f0dcc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:04.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0de2ffd700 1 -- 172.21.15.118:0/2161989972 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f0ddc07e4c0 con 0x7f0dec06e4c0 2024-08-18T13:53:04.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.877+0000 7f0de3fff700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dcc060890 0x7f0dcc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:04.883 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.881+0000 7f0de3fff700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dcc060890 0x7f0dcc062d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0c5720 tx=0x7f0dd4007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:04.889 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:04.885+0000 7f0de2ffd700 1 -- 172.21.15.118:0/2161989972 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0dec138810 con 0x7f0dec06e4c0 2024-08-18T13:53:05.054 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.053+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 --> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0dec138810 con 0x7f0dcc060890 2024-08-18T13:53:05.071 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.069+0000 7f0de2ffd700 1 -- 172.21.15.118:0/2161989972 <== mgr.4110 v2:172.21.15.118:6832/24192 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+375657 (secure 0 0 0) 0x7f0dec138810 con 0x7f0dcc060890 2024-08-18T13:53:05.078 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:53:05.083 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dcc060890 msgr2=0x7f0dcc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dcc060890 0x7f0dcc062d40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0c5720 tx=0x7f0dd4007d50 comp rx=0 tx=0).stop 2024-08-18T13:53:05.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec06e4c0 msgr2=0x7f0dec14a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec06e4c0 0x7f0dec14a4b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009f80 tx=0x7f0ddc017000 comp rx=0 tx=0).stop 2024-08-18T13:53:05.084 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 shutdown_connections 2024-08-18T13:53:05.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0dcc060890 0x7f0dcc062d40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0dec0c22a0 0x7f0dec138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0dec0be5a0 0x7f0dec14a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.085 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 --2- 172.21.15.118:0/2161989972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dec06e4c0 0x7f0dec14a4b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 >> 172.21.15.118:0/2161989972 conn(0x7f0dec06c020 msgr2=0x7f0dec151060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:05.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 shutdown_connections 2024-08-18T13:53:05.086 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.073+0000 7f0df0837700 1 -- 172.21.15.118:0/2161989972 wait complete. 2024-08-18T13:53:05.087 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-08-18T13:53:05.090 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-08-18T13:53:03.514836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1123881704,"num_objects":11047,"num_object_clones":0,"num_object_copies":43906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11047,"num_whiteouts":0,"num_read":22597,"num_read_kb":1390334,"num_write":28078,"num_write_kb":1250468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576095,"num_omap_keys":1100,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33897,"ondisk_log_size":33897,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4738080,"kb_used_data":2321824,"kb_used_omap":13,"kb_used_meta":2416178,"kb_avail":750236640,"statfs":{"total":773094113280,"available":768242319360,"internally_reserved":0,"allocated":2377547776,"data_stored":2375144936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2474166976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005213"},"pg_stats":[{"pgid":"2.1f","version":"54'156","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.111340+0000","last_change":"2024-08-18T13:52:07.111260+0000","last_active":"2024-08-18T13:52:07.111340+0000","last_peered":"2024-08-18T13:52:07.111340+0000","last_clean":"2024-08-18T13:52:07.111340+0000","last_became_active":"2024-08-18T13:39:34.395910+0000","last_became_peered":"2024-08-18T13:39:34.395910+0000","last_unstale":"2024-08-18T13:52:07.111340+0000","last_undegraded":"2024-08-18T13:52:07.111340+0000","last_fullsized":"2024-08-18T13:52:07.111340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'156","last_scrub_stamp":"2024-08-18T13:52:07.111121+0000","last_deep_scrub":"54'156","last_deep_scrub_stamp":"2024-08-18T13:52:07.111121+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.111121+0000","objects_scrubbed":5,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:38.124367+0000","scrub_duration":0.025498494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389178,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":28696,"num_write":150,"num_write_kb":8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11702,"num_omap_keys":22,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.258859+0000","last_change":"2024-08-18T13:52:27.258755+0000","last_active":"2024-08-18T13:52:27.258859+0000","last_peered":"2024-08-18T13:52:27.258859+0000","last_clean":"2024-08-18T13:52:27.258859+0000","last_became_active":"2024-08-18T13:39:35.405650+0000","last_became_peered":"2024-08-18T13:39:35.405650+0000","last_unstale":"2024-08-18T13:52:27.258859+0000","last_undegraded":"2024-08-18T13:52:27.258859+0000","last_fullsized":"2024-08-18T13:52:27.258859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:27.258561+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:27.258561+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.258561+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:05.192013+0000","scrub_duration":0.0038873929999999998,"objects_trimmed":0,"snaptrim_duration":0.0064850389999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"54'676","reported_seq":3774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:55.288189+0000","last_change":"2024-08-18T13:52:55.288117+0000","last_active":"2024-08-18T13:52:55.288189+0000","last_peered":"2024-08-18T13:52:55.288189+0000","last_clean":"2024-08-18T13:52:55.288189+0000","last_became_active":"2024-08-18T13:39:39.453126+0000","last_became_peered":"2024-08-18T13:39:39.453126+0000","last_unstale":"2024-08-18T13:52:55.288189+0000","last_undegraded":"2024-08-18T13:52:55.288189+0000","last_fullsized":"2024-08-18T13:52:55.288189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'676","last_scrub_stamp":"2024-08-18T13:52:55.287980+0000","last_deep_scrub":"54'676","last_deep_scrub_stamp":"2024-08-18T13:52:55.287980+0000","last_clean_scrub_stamp":"2024-08-18T13:52:55.287980+0000","objects_scrubbed":316,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:36:44.156724+0000","scrub_duration":0.12964363200000001,"objects_trimmed":0,"snaptrim_duration":0.0067303959999999996,"stat_sum":{"num_bytes":37566158,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":1013,"num_read_kb":36248,"num_write":673,"num_write_kb":41139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"54'79","reported_seq":138,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.012943+0000","last_change":"2024-08-18T13:52:14.012891+0000","last_active":"2024-08-18T13:52:14.012943+0000","last_peered":"2024-08-18T13:52:14.012943+0000","last_clean":"2024-08-18T13:52:14.012943+0000","last_became_active":"2024-08-18T13:39:34.391411+0000","last_became_peered":"2024-08-18T13:39:34.391411+0000","last_unstale":"2024-08-18T13:52:14.012943+0000","last_undegraded":"2024-08-18T13:52:14.012943+0000","last_fullsized":"2024-08-18T13:52:14.012943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'79","last_scrub_stamp":"2024-08-18T13:52:14.012738+0000","last_deep_scrub":"54'79","last_deep_scrub_stamp":"2024-08-18T13:52:14.012738+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.012738+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:27:28.341973+0000","scrub_duration":0.015505003999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":75,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"54'87","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:01.076425+0000","last_change":"2024-08-18T13:52:01.076369+0000","last_active":"2024-08-18T13:52:01.076425+0000","last_peered":"2024-08-18T13:52:01.076425+0000","last_clean":"2024-08-18T13:52:01.076425+0000","last_became_active":"2024-08-18T13:39:35.408649+0000","last_became_peered":"2024-08-18T13:39:35.408649+0000","last_unstale":"2024-08-18T13:52:01.076425+0000","last_undegraded":"2024-08-18T13:52:01.076425+0000","last_fullsized":"2024-08-18T13:52:01.076425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:01.076152+0000","last_clean_scrub_stamp":"2024-08-18T13:52:01.076152+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:46:50.073762+0000","scrub_duration":0.0039656070000000003,"objects_trimmed":0,"snaptrim_duration":0.0075430439999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"54'705","reported_seq":3584,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.827339+0000","last_change":"2024-08-18T13:52:18.827164+0000","last_active":"2024-08-18T13:52:18.827339+0000","last_peered":"2024-08-18T13:52:18.827339+0000","last_clean":"2024-08-18T13:52:18.827339+0000","last_became_active":"2024-08-18T13:39:39.444982+0000","last_became_peered":"2024-08-18T13:39:39.444982+0000","last_unstale":"2024-08-18T13:52:18.827339+0000","last_undegraded":"2024-08-18T13:52:18.827339+0000","last_fullsized":"2024-08-18T13:52:18.827339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'705","last_scrub_stamp":"2024-08-18T13:52:18.826906+0000","last_deep_scrub":"54'705","last_deep_scrub_stamp":"2024-08-18T13:52:18.826906+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.826906+0000","objects_scrubbed":370,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:26:33.481234+0000","scrub_duration":0.157720147,"objects_trimmed":0,"snaptrim_duration":0.0075371290000000001,"stat_sum":{"num_bytes":29851183,"num_objects":370,"num_object_clones":0,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":792,"num_read_kb":20881,"num_write":708,"num_write_kb":37566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"54'202","reported_seq":278,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.924150+0000","last_change":"2024-08-18T13:52:22.924101+0000","last_active":"2024-08-18T13:52:22.924150+0000","last_peered":"2024-08-18T13:52:22.924150+0000","last_clean":"2024-08-18T13:52:22.924150+0000","last_became_active":"2024-08-18T13:39:34.394430+0000","last_became_peered":"2024-08-18T13:39:34.394430+0000","last_unstale":"2024-08-18T13:52:22.924150+0000","last_undegraded":"2024-08-18T13:52:22.924150+0000","last_fullsized":"2024-08-18T13:52:22.924150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'202","last_scrub_stamp":"2024-08-18T13:52:22.923937+0000","last_deep_scrub":"54'202","last_deep_scrub_stamp":"2024-08-18T13:52:22.923937+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.923937+0000","objects_scrubbed":8,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:49.870790+0000","scrub_duration":0.033031854999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":69650,"num_write":192,"num_write_kb":12515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31566,"num_omap_keys":60,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.557506+0000","last_change":"2024-08-18T13:52:27.557449+0000","last_active":"2024-08-18T13:52:27.557506+0000","last_peered":"2024-08-18T13:52:27.557506+0000","last_clean":"2024-08-18T13:52:27.557506+0000","last_became_active":"2024-08-18T13:39:35.398912+0000","last_became_peered":"2024-08-18T13:39:35.398912+0000","last_unstale":"2024-08-18T13:52:27.557506+0000","last_undegraded":"2024-08-18T13:52:27.557506+0000","last_fullsized":"2024-08-18T13:52:27.557506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:27.557280+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:27.557280+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.557280+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:04.966509+0000","scrub_duration":0.003068201,"objects_trimmed":0,"snaptrim_duration":0.0046085359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"54'680","reported_seq":3503,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:45.654580+0000","last_change":"2024-08-18T13:52:45.654513+0000","last_active":"2024-08-18T13:52:45.654580+0000","last_peered":"2024-08-18T13:52:45.654580+0000","last_clean":"2024-08-18T13:52:45.654580+0000","last_became_active":"2024-08-18T13:40:16.608743+0000","last_became_peered":"2024-08-18T13:40:16.608743+0000","last_unstale":"2024-08-18T13:52:45.654580+0000","last_undegraded":"2024-08-18T13:52:45.654580+0000","last_fullsized":"2024-08-18T13:52:45.654580+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'680","last_scrub_stamp":"2024-08-18T13:52:45.654361+0000","last_deep_scrub":"54'680","last_deep_scrub_stamp":"2024-08-18T13:52:45.654361+0000","last_clean_scrub_stamp":"2024-08-18T13:52:45.654361+0000","objects_scrubbed":318,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:06:54.291684+0000","scrub_duration":0.14331348099999999,"objects_trimmed":0,"snaptrim_duration":0.0080275980000000004,"stat_sum":{"num_bytes":28774140,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":727,"num_read_kb":13856,"num_write":680,"num_write_kb":28426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"54'27","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.933094+0000","last_change":"2024-08-18T13:52:15.933048+0000","last_active":"2024-08-18T13:52:15.933094+0000","last_peered":"2024-08-18T13:52:15.933094+0000","last_clean":"2024-08-18T13:52:15.933094+0000","last_became_active":"2024-08-18T13:39:34.394308+0000","last_became_peered":"2024-08-18T13:39:34.394308+0000","last_unstale":"2024-08-18T13:52:15.933094+0000","last_undegraded":"2024-08-18T13:52:15.933094+0000","last_fullsized":"2024-08-18T13:52:15.933094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'27","last_scrub_stamp":"2024-08-18T13:52:15.932932+0000","last_deep_scrub":"54'27","last_deep_scrub_stamp":"2024-08-18T13:52:15.932932+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.932932+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:50:28.024428+0000","scrub_duration":0.016823828999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":25,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12356,"num_omap_keys":25,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"54'87","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.530882+0000","last_change":"2024-08-18T13:52:31.530815+0000","last_active":"2024-08-18T13:52:31.530882+0000","last_peered":"2024-08-18T13:52:31.530882+0000","last_clean":"2024-08-18T13:52:31.530882+0000","last_became_active":"2024-08-18T13:39:35.399758+0000","last_became_peered":"2024-08-18T13:39:35.399758+0000","last_unstale":"2024-08-18T13:52:31.530882+0000","last_undegraded":"2024-08-18T13:52:31.530882+0000","last_fullsized":"2024-08-18T13:52:31.530882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:31.530605+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:31.530605+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.530605+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:47:04.712973+0000","scrub_duration":0.0031435159999999998,"objects_trimmed":0,"snaptrim_duration":0.0065299970000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"54'674","reported_seq":2926,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.822593+0000","last_change":"2024-08-18T13:52:29.822494+0000","last_active":"2024-08-18T13:52:29.822593+0000","last_peered":"2024-08-18T13:52:29.822593+0000","last_clean":"2024-08-18T13:52:29.822593+0000","last_became_active":"2024-08-18T13:39:39.435584+0000","last_became_peered":"2024-08-18T13:39:39.435584+0000","last_unstale":"2024-08-18T13:52:29.822593+0000","last_undegraded":"2024-08-18T13:52:29.822593+0000","last_fullsized":"2024-08-18T13:52:29.822593+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'674","last_scrub_stamp":"2024-08-18T13:52:29.822290+0000","last_deep_scrub":"54'674","last_deep_scrub_stamp":"2024-08-18T13:52:29.822290+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.822290+0000","objects_scrubbed":345,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:23:06.162523+0000","scrub_duration":0.13583155499999999,"objects_trimmed":0,"snaptrim_duration":0.0078514629999999995,"stat_sum":{"num_bytes":11851705,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":166,"num_read_kb":1355,"num_write":673,"num_write_kb":11932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:24.298191+0000","last_change":"2024-08-18T13:52:24.298114+0000","last_active":"2024-08-18T13:52:24.298191+0000","last_peered":"2024-08-18T13:52:24.298191+0000","last_clean":"2024-08-18T13:52:24.298191+0000","last_became_active":"2024-08-18T13:39:35.408623+0000","last_became_peered":"2024-08-18T13:39:35.408623+0000","last_unstale":"2024-08-18T13:52:24.298191+0000","last_undegraded":"2024-08-18T13:52:24.298191+0000","last_fullsized":"2024-08-18T13:52:24.298191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:24.297918+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:24.297918+0000","last_clean_scrub_stamp":"2024-08-18T13:52:24.297918+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:46:26.475472+0000","scrub_duration":0.0035255989999999999,"objects_trimmed":0,"snaptrim_duration":0.0033699279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"54'252","reported_seq":322,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.940516+0000","last_change":"2024-08-18T13:52:21.940455+0000","last_active":"2024-08-18T13:52:21.940516+0000","last_peered":"2024-08-18T13:52:21.940516+0000","last_clean":"2024-08-18T13:52:21.940516+0000","last_became_active":"2024-08-18T13:39:34.394660+0000","last_became_peered":"2024-08-18T13:39:34.394660+0000","last_unstale":"2024-08-18T13:52:21.940516+0000","last_undegraded":"2024-08-18T13:52:21.940516+0000","last_fullsized":"2024-08-18T13:52:21.940516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'252","last_scrub_stamp":"2024-08-18T13:52:21.940309+0000","last_deep_scrub":"54'252","last_deep_scrub_stamp":"2024-08-18T13:52:21.940309+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.940309+0000","objects_scrubbed":9,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:44:34.338028+0000","scrub_duration":0.033709736999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12304619,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":23,"num_read_kb":45068,"num_write":239,"num_write_kb":12181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43761,"num_omap_keys":83,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"54'638","reported_seq":3304,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:43.712787+0000","last_change":"2024-08-18T13:52:43.712687+0000","last_active":"2024-08-18T13:52:43.712787+0000","last_peered":"2024-08-18T13:52:43.712787+0000","last_clean":"2024-08-18T13:52:43.712787+0000","last_became_active":"2024-08-18T13:39:39.439197+0000","last_became_peered":"2024-08-18T13:39:39.439197+0000","last_unstale":"2024-08-18T13:52:43.712787+0000","last_undegraded":"2024-08-18T13:52:43.712787+0000","last_fullsized":"2024-08-18T13:52:43.712787+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'638","last_scrub_stamp":"2024-08-18T13:52:43.712493+0000","last_deep_scrub":"54'638","last_deep_scrub_stamp":"2024-08-18T13:52:43.712493+0000","last_clean_scrub_stamp":"2024-08-18T13:52:43.712493+0000","objects_scrubbed":307,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:17:02.878482+0000","scrub_duration":0.127629347,"objects_trimmed":0,"snaptrim_duration":0.0059019540000000001,"stat_sum":{"num_bytes":16782769,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":581,"num_read_kb":14450,"num_write":639,"num_write_kb":24822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"54'87","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.890088+0000","last_change":"2024-08-18T13:52:08.890024+0000","last_active":"2024-08-18T13:52:08.890088+0000","last_peered":"2024-08-18T13:52:08.890088+0000","last_clean":"2024-08-18T13:52:08.890088+0000","last_became_active":"2024-08-18T13:39:35.407649+0000","last_became_peered":"2024-08-18T13:39:35.407649+0000","last_unstale":"2024-08-18T13:52:08.890088+0000","last_undegraded":"2024-08-18T13:52:08.890088+0000","last_fullsized":"2024-08-18T13:52:08.890088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:08.889811+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:08.889811+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.889811+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:42:55.362427+0000","scrub_duration":0.0034394989999999999,"objects_trimmed":0,"snaptrim_duration":0.0074252320000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"54'32","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:11.142035+0000","last_change":"2024-08-18T13:52:11.141993+0000","last_active":"2024-08-18T13:52:11.142035+0000","last_peered":"2024-08-18T13:52:11.142035+0000","last_clean":"2024-08-18T13:52:11.142035+0000","last_became_active":"2024-08-18T13:39:34.393539+0000","last_became_peered":"2024-08-18T13:39:34.393539+0000","last_unstale":"2024-08-18T13:52:11.142035+0000","last_undegraded":"2024-08-18T13:52:11.142035+0000","last_fullsized":"2024-08-18T13:52:11.142035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'32","last_scrub_stamp":"2024-08-18T13:52:11.141864+0000","last_deep_scrub":"54'32","last_deep_scrub_stamp":"2024-08-18T13:52:11.141864+0000","last_clean_scrub_stamp":"2024-08-18T13:52:11.141864+0000","objects_scrubbed":4,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:40.879760+0000","scrub_duration":0.016960533999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":28,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28048,"num_omap_keys":55,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"54'709","reported_seq":3823,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:35.849372+0000","last_change":"2024-08-18T13:52:35.849308+0000","last_active":"2024-08-18T13:52:35.849372+0000","last_peered":"2024-08-18T13:52:35.849372+0000","last_clean":"2024-08-18T13:52:35.849372+0000","last_became_active":"2024-08-18T13:39:39.439395+0000","last_became_peered":"2024-08-18T13:39:39.439395+0000","last_unstale":"2024-08-18T13:52:35.849372+0000","last_undegraded":"2024-08-18T13:52:35.849372+0000","last_fullsized":"2024-08-18T13:52:35.849372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'709","last_scrub_stamp":"2024-08-18T13:52:35.849145+0000","last_deep_scrub":"54'709","last_deep_scrub_stamp":"2024-08-18T13:52:35.849145+0000","last_clean_scrub_stamp":"2024-08-18T13:52:35.849145+0000","objects_scrubbed":347,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:20:01.973785+0000","scrub_duration":0.151375125,"objects_trimmed":0,"snaptrim_duration":0.0057029610000000003,"stat_sum":{"num_bytes":33413248,"num_objects":347,"num_object_clones":0,"num_object_copies":1388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":1028,"num_read_kb":27410,"num_write":707,"num_write_kb":41185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:28.273167+0000","last_change":"2024-08-18T13:52:28.273103+0000","last_active":"2024-08-18T13:52:28.273167+0000","last_peered":"2024-08-18T13:52:28.273167+0000","last_clean":"2024-08-18T13:52:28.273167+0000","last_became_active":"2024-08-18T13:39:35.402947+0000","last_became_peered":"2024-08-18T13:39:35.402947+0000","last_unstale":"2024-08-18T13:52:28.273167+0000","last_undegraded":"2024-08-18T13:52:28.273167+0000","last_fullsized":"2024-08-18T13:52:28.273167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:28.272879+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:28.272879+0000","last_clean_scrub_stamp":"2024-08-18T13:52:28.272879+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:03.396288+0000","scrub_duration":0.0035927960000000001,"objects_trimmed":0,"snaptrim_duration":0.0069243719999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"54'14","reported_seq":69,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:06.184482+0000","last_change":"2024-08-18T13:52:06.184363+0000","last_active":"2024-08-18T13:52:06.184482+0000","last_peered":"2024-08-18T13:52:06.184482+0000","last_clean":"2024-08-18T13:52:06.184482+0000","last_became_active":"2024-08-18T13:39:34.389373+0000","last_became_peered":"2024-08-18T13:39:34.389373+0000","last_unstale":"2024-08-18T13:52:06.184482+0000","last_undegraded":"2024-08-18T13:52:06.184482+0000","last_fullsized":"2024-08-18T13:52:06.184482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'14","last_scrub_stamp":"2024-08-18T13:52:06.184119+0000","last_deep_scrub":"54'14","last_deep_scrub_stamp":"2024-08-18T13:52:06.184119+0000","last_clean_scrub_stamp":"2024-08-18T13:52:06.184119+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:52.279744+0000","scrub_duration":0.013536823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":11,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'683","reported_seq":3594,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:43.210788+0000","last_change":"2024-08-18T13:52:43.210717+0000","last_active":"2024-08-18T13:52:43.210788+0000","last_peered":"2024-08-18T13:52:43.210788+0000","last_clean":"2024-08-18T13:52:43.210788+0000","last_became_active":"2024-08-18T13:39:39.456809+0000","last_became_peered":"2024-08-18T13:39:39.456809+0000","last_unstale":"2024-08-18T13:52:43.210788+0000","last_undegraded":"2024-08-18T13:52:43.210788+0000","last_fullsized":"2024-08-18T13:52:43.210788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'683","last_scrub_stamp":"2024-08-18T13:52:43.210556+0000","last_deep_scrub":"54'683","last_deep_scrub_stamp":"2024-08-18T13:52:43.210556+0000","last_clean_scrub_stamp":"2024-08-18T13:52:43.210556+0000","objects_scrubbed":349,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:12:13.018315+0000","scrub_duration":0.13235233399999999,"objects_trimmed":0,"snaptrim_duration":0.0079287170000000001,"stat_sum":{"num_bytes":20263748,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":825,"num_read_kb":25932,"num_write":679,"num_write_kb":28331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"54'92","reported_seq":2167,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.649198+0000","last_change":"2024-08-18T13:52:10.649127+0000","last_active":"2024-08-18T13:52:10.649198+0000","last_peered":"2024-08-18T13:52:10.649198+0000","last_clean":"2024-08-18T13:52:10.649198+0000","last_became_active":"2024-08-18T13:39:35.401349+0000","last_became_peered":"2024-08-18T13:39:35.401349+0000","last_unstale":"2024-08-18T13:52:10.649198+0000","last_undegraded":"2024-08-18T13:52:10.649198+0000","last_fullsized":"2024-08-18T13:52:10.649198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:10.648930+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:10.648930+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.648930+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:46:12.826484+0000","scrub_duration":0.0036575269999999998,"objects_trimmed":0,"snaptrim_duration":0.0068791520000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"54'110","reported_seq":168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.973413+0000","last_change":"2024-08-18T13:52:18.973337+0000","last_active":"2024-08-18T13:52:18.973413+0000","last_peered":"2024-08-18T13:52:18.973413+0000","last_clean":"2024-08-18T13:52:18.973413+0000","last_became_active":"2024-08-18T13:39:34.394077+0000","last_became_peered":"2024-08-18T13:39:34.394077+0000","last_unstale":"2024-08-18T13:52:18.973413+0000","last_undegraded":"2024-08-18T13:52:18.973413+0000","last_fullsized":"2024-08-18T13:52:18.973413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'110","last_scrub_stamp":"2024-08-18T13:52:18.973134+0000","last_deep_scrub":"54'110","last_deep_scrub_stamp":"2024-08-18T13:52:18.973134+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.973134+0000","objects_scrubbed":4,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:40.390138+0000","scrub_duration":0.045608213000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":108,"num_write_kb":4251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12312,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"54'617","reported_seq":3476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:41.711739+0000","last_change":"2024-08-18T13:52:41.711684+0000","last_active":"2024-08-18T13:52:41.711739+0000","last_peered":"2024-08-18T13:52:41.711739+0000","last_clean":"2024-08-18T13:52:41.711739+0000","last_became_active":"2024-08-18T13:39:39.443027+0000","last_became_peered":"2024-08-18T13:39:39.443027+0000","last_unstale":"2024-08-18T13:52:41.711739+0000","last_undegraded":"2024-08-18T13:52:41.711739+0000","last_fullsized":"2024-08-18T13:52:41.711739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'617","last_scrub_stamp":"2024-08-18T13:52:41.711559+0000","last_deep_scrub":"54'617","last_deep_scrub_stamp":"2024-08-18T13:52:41.711559+0000","last_clean_scrub_stamp":"2024-08-18T13:52:41.711559+0000","objects_scrubbed":338,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:19:56.474223+0000","scrub_duration":0.150787001,"objects_trimmed":0,"snaptrim_duration":0.004078028,"stat_sum":{"num_bytes":24272349,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":773,"num_read_kb":25488,"num_write":617,"num_write_kb":28093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"54'93","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.019178+0000","last_change":"2024-08-18T13:52:09.019114+0000","last_active":"2024-08-18T13:52:09.019178+0000","last_peered":"2024-08-18T13:52:09.019178+0000","last_clean":"2024-08-18T13:52:09.019178+0000","last_became_active":"2024-08-18T13:39:35.407576+0000","last_became_peered":"2024-08-18T13:39:35.407576+0000","last_unstale":"2024-08-18T13:52:09.019178+0000","last_undegraded":"2024-08-18T13:52:09.019178+0000","last_fullsized":"2024-08-18T13:52:09.019178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'93","last_scrub_stamp":"2024-08-18T13:52:09.018858+0000","last_deep_scrub":"54'93","last_deep_scrub_stamp":"2024-08-18T13:52:09.018858+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.018858+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:55.114483+0000","scrub_duration":0.0031236559999999998,"objects_trimmed":0,"snaptrim_duration":0.0060661889999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"54'114","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.265449+0000","last_change":"2024-08-18T13:52:05.265412+0000","last_active":"2024-08-18T13:52:05.265449+0000","last_peered":"2024-08-18T13:52:05.265449+0000","last_clean":"2024-08-18T13:52:05.265449+0000","last_became_active":"2024-08-18T13:39:34.395270+0000","last_became_peered":"2024-08-18T13:39:34.395270+0000","last_unstale":"2024-08-18T13:52:05.265449+0000","last_undegraded":"2024-08-18T13:52:05.265449+0000","last_fullsized":"2024-08-18T13:52:05.265449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'114","last_scrub_stamp":"2024-08-18T13:52:05.265313+0000","last_deep_scrub":"54'114","last_deep_scrub_stamp":"2024-08-18T13:52:05.265313+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.265313+0000","objects_scrubbed":4,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:57:48.580753+0000","scrub_duration":0.015757296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":111,"num_write_kb":4166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16148,"num_omap_keys":31,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"54'720","reported_seq":2994,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.759072+0000","last_change":"2024-08-18T13:52:21.758999+0000","last_active":"2024-08-18T13:52:21.759072+0000","last_peered":"2024-08-18T13:52:21.759072+0000","last_clean":"2024-08-18T13:52:21.759072+0000","last_became_active":"2024-08-18T13:39:39.452590+0000","last_became_peered":"2024-08-18T13:39:39.452590+0000","last_unstale":"2024-08-18T13:52:21.759072+0000","last_undegraded":"2024-08-18T13:52:21.759072+0000","last_fullsized":"2024-08-18T13:52:21.759072+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'720","last_scrub_stamp":"2024-08-18T13:52:21.758840+0000","last_deep_scrub":"54'720","last_deep_scrub_stamp":"2024-08-18T13:52:21.758840+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.758840+0000","objects_scrubbed":368,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:36.545679+0000","scrub_duration":0.120970787,"objects_trimmed":0,"snaptrim_duration":0.0079796720000000002,"stat_sum":{"num_bytes":17871290,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":187,"num_read_kb":7768,"num_write":717,"num_write_kb":21887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:26.213190+0000","last_change":"2024-08-18T13:52:26.213112+0000","last_active":"2024-08-18T13:52:26.213190+0000","last_peered":"2024-08-18T13:52:26.213190+0000","last_clean":"2024-08-18T13:52:26.213190+0000","last_became_active":"2024-08-18T13:39:35.404162+0000","last_became_peered":"2024-08-18T13:39:35.404162+0000","last_unstale":"2024-08-18T13:52:26.213190+0000","last_undegraded":"2024-08-18T13:52:26.213190+0000","last_fullsized":"2024-08-18T13:52:26.213190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:26.212910+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:26.212910+0000","last_clean_scrub_stamp":"2024-08-18T13:52:26.212910+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:34.964479+0000","scrub_duration":0.0037999150000000001,"objects_trimmed":0,"snaptrim_duration":0.0054340660000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"54'67","reported_seq":128,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.141636+0000","last_change":"2024-08-18T13:52:07.141608+0000","last_active":"2024-08-18T13:52:07.141636+0000","last_peered":"2024-08-18T13:52:07.141636+0000","last_clean":"2024-08-18T13:52:07.141636+0000","last_became_active":"2024-08-18T13:39:34.388629+0000","last_became_peered":"2024-08-18T13:39:34.388629+0000","last_unstale":"2024-08-18T13:52:07.141636+0000","last_undegraded":"2024-08-18T13:52:07.141636+0000","last_fullsized":"2024-08-18T13:52:07.141636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'67","last_scrub_stamp":"2024-08-18T13:52:07.141528+0000","last_deep_scrub":"54'67","last_deep_scrub_stamp":"2024-08-18T13:52:07.141528+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.141528+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:03.991301+0000","scrub_duration":0.018011553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":184,"num_write":164,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23450,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"54'714","reported_seq":2954,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:51.348574+0000","last_change":"2024-08-18T13:52:51.348337+0000","last_active":"2024-08-18T13:52:51.348574+0000","last_peered":"2024-08-18T13:52:51.348574+0000","last_clean":"2024-08-18T13:52:51.348574+0000","last_became_active":"2024-08-18T13:39:39.449211+0000","last_became_peered":"2024-08-18T13:39:39.449211+0000","last_unstale":"2024-08-18T13:52:51.348574+0000","last_undegraded":"2024-08-18T13:52:51.348574+0000","last_fullsized":"2024-08-18T13:52:51.348574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'714","last_scrub_stamp":"2024-08-18T13:52:51.348200+0000","last_deep_scrub":"54'714","last_deep_scrub_stamp":"2024-08-18T13:52:51.348200+0000","last_clean_scrub_stamp":"2024-08-18T13:52:51.348200+0000","objects_scrubbed":368,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:36:17.966062+0000","scrub_duration":0.123053027,"objects_trimmed":0,"snaptrim_duration":0.0057306029999999999,"stat_sum":{"num_bytes":13242151,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":153,"num_read_kb":5341,"num_write":712,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.019415+0000","last_change":"2024-08-18T13:52:20.019359+0000","last_active":"2024-08-18T13:52:20.019415+0000","last_peered":"2024-08-18T13:52:20.019415+0000","last_clean":"2024-08-18T13:52:20.019415+0000","last_became_active":"2024-08-18T13:39:35.400293+0000","last_became_peered":"2024-08-18T13:39:35.400293+0000","last_unstale":"2024-08-18T13:52:20.019415+0000","last_undegraded":"2024-08-18T13:52:20.019415+0000","last_fullsized":"2024-08-18T13:52:20.019415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:20.019175+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:20.019175+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.019175+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:46:53.201543+0000","scrub_duration":0.0031831339999999998,"objects_trimmed":0,"snaptrim_duration":0.0064599770000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"54'292","reported_seq":351,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.937801+0000","last_change":"2024-08-18T13:52:16.937745+0000","last_active":"2024-08-18T13:52:16.937801+0000","last_peered":"2024-08-18T13:52:16.937801+0000","last_clean":"2024-08-18T13:52:16.937801+0000","last_became_active":"2024-08-18T13:39:34.394846+0000","last_became_peered":"2024-08-18T13:39:34.394846+0000","last_unstale":"2024-08-18T13:52:16.937801+0000","last_undegraded":"2024-08-18T13:52:16.937801+0000","last_fullsized":"2024-08-18T13:52:16.937801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'292","last_scrub_stamp":"2024-08-18T13:52:16.937598+0000","last_deep_scrub":"54'292","last_deep_scrub_stamp":"2024-08-18T13:52:16.937598+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.937598+0000","objects_scrubbed":3,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:46:56.453503+0000","scrub_duration":0.022315498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":12176,"num_write":284,"num_write_kb":8352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8544,"num_omap_keys":16,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"54'704","reported_seq":3449,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:47.413468+0000","last_change":"2024-08-18T13:52:47.413387+0000","last_active":"2024-08-18T13:52:47.413468+0000","last_peered":"2024-08-18T13:52:47.413468+0000","last_clean":"2024-08-18T13:52:47.413468+0000","last_became_active":"2024-08-18T13:39:39.453138+0000","last_became_peered":"2024-08-18T13:39:39.453138+0000","last_unstale":"2024-08-18T13:52:47.413468+0000","last_undegraded":"2024-08-18T13:52:47.413468+0000","last_fullsized":"2024-08-18T13:52:47.413468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'704","last_scrub_stamp":"2024-08-18T13:52:47.413204+0000","last_deep_scrub":"54'704","last_deep_scrub_stamp":"2024-08-18T13:52:47.413204+0000","last_clean_scrub_stamp":"2024-08-18T13:52:47.413204+0000","objects_scrubbed":340,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:12:17.220963+0000","scrub_duration":0.14654446300000001,"objects_trimmed":0,"snaptrim_duration":0.0060004689999999996,"stat_sum":{"num_bytes":28446730,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":659,"num_read_kb":21792,"num_write":702,"num_write_kb":28155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"54'84","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.470644+0000","last_change":"2024-08-18T13:52:21.470559+0000","last_active":"2024-08-18T13:52:21.470644+0000","last_peered":"2024-08-18T13:52:21.470644+0000","last_clean":"2024-08-18T13:52:21.470644+0000","last_became_active":"2024-08-18T13:39:35.409802+0000","last_became_peered":"2024-08-18T13:39:35.409802+0000","last_unstale":"2024-08-18T13:52:21.470644+0000","last_undegraded":"2024-08-18T13:52:21.470644+0000","last_fullsized":"2024-08-18T13:52:21.470644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'84","last_scrub_stamp":"2024-08-18T13:52:21.470358+0000","last_deep_scrub":"54'84","last_deep_scrub_stamp":"2024-08-18T13:52:21.470358+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.470358+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:32.524505+0000","scrub_duration":0.0039169649999999997,"objects_trimmed":0,"snaptrim_duration":0.006455484,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"54'660","reported_seq":714,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.030412+0000","last_change":"2024-08-18T13:52:16.030333+0000","last_active":"2024-08-18T13:52:16.030412+0000","last_peered":"2024-08-18T13:52:16.030412+0000","last_clean":"2024-08-18T13:52:16.030412+0000","last_became_active":"2024-08-18T13:39:34.379865+0000","last_became_peered":"2024-08-18T13:39:34.379865+0000","last_unstale":"2024-08-18T13:52:16.030412+0000","last_undegraded":"2024-08-18T13:52:16.030412+0000","last_fullsized":"2024-08-18T13:52:16.030412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'660","last_scrub_stamp":"2024-08-18T13:52:16.030100+0000","last_deep_scrub":"54'660","last_deep_scrub_stamp":"2024-08-18T13:52:16.030100+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.030100+0000","objects_scrubbed":3,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:55:53.439330+0000","scrub_duration":0.021515340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24518,"num_omap_keys":46,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"54'691","reported_seq":3367,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.103051+0000","last_change":"2024-08-18T13:52:31.102998+0000","last_active":"2024-08-18T13:52:31.103051+0000","last_peered":"2024-08-18T13:52:31.103051+0000","last_clean":"2024-08-18T13:52:31.103051+0000","last_became_active":"2024-08-18T13:39:39.438794+0000","last_became_peered":"2024-08-18T13:39:39.438794+0000","last_unstale":"2024-08-18T13:52:31.103051+0000","last_undegraded":"2024-08-18T13:52:31.103051+0000","last_fullsized":"2024-08-18T13:52:31.103051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'691","last_scrub_stamp":"2024-08-18T13:52:31.102865+0000","last_deep_scrub":"54'691","last_deep_scrub_stamp":"2024-08-18T13:52:31.102865+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.102865+0000","objects_scrubbed":341,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:16:32.971846+0000","scrub_duration":0.16053246500000001,"objects_trimmed":0,"snaptrim_duration":0.0067583870000000002,"stat_sum":{"num_bytes":38683931,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":590,"num_read_kb":16518,"num_write":694,"num_write_kb":42167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"54'97","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.632549+0000","last_change":"2024-08-18T13:52:08.632437+0000","last_active":"2024-08-18T13:52:08.632549+0000","last_peered":"2024-08-18T13:52:08.632549+0000","last_clean":"2024-08-18T13:52:08.632549+0000","last_became_active":"2024-08-18T13:39:35.407542+0000","last_became_peered":"2024-08-18T13:39:35.407542+0000","last_unstale":"2024-08-18T13:52:08.632549+0000","last_undegraded":"2024-08-18T13:52:08.632549+0000","last_fullsized":"2024-08-18T13:52:08.632549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'97","last_scrub_stamp":"2024-08-18T13:52:08.632220+0000","last_deep_scrub":"54'97","last_deep_scrub_stamp":"2024-08-18T13:52:08.632220+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.632220+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:05:30.186903+0000","scrub_duration":0.0038014519999999999,"objects_trimmed":0,"snaptrim_duration":0.005854756,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"54'387","reported_seq":453,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.408711+0000","last_change":"2024-08-18T13:52:09.408667+0000","last_active":"2024-08-18T13:52:09.408711+0000","last_peered":"2024-08-18T13:52:09.408711+0000","last_clean":"2024-08-18T13:52:09.408711+0000","last_became_active":"2024-08-18T13:39:34.380723+0000","last_became_peered":"2024-08-18T13:39:34.380723+0000","last_unstale":"2024-08-18T13:52:09.408711+0000","last_undegraded":"2024-08-18T13:52:09.408711+0000","last_fullsized":"2024-08-18T13:52:09.408711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'387","last_scrub_stamp":"2024-08-18T13:52:09.408576+0000","last_deep_scrub":"54'387","last_deep_scrub_stamp":"2024-08-18T13:52:09.408576+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.408576+0000","objects_scrubbed":4,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:46:58.406186+0000","scrub_duration":0.023922603000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":43191,"num_write":382,"num_write_kb":8485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16508,"num_omap_keys":31,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"54'661","reported_seq":4484,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:35.155057+0000","last_change":"2024-08-18T13:52:35.155004+0000","last_active":"2024-08-18T13:52:35.155057+0000","last_peered":"2024-08-18T13:52:35.155057+0000","last_clean":"2024-08-18T13:52:35.155057+0000","last_became_active":"2024-08-18T13:39:39.441996+0000","last_became_peered":"2024-08-18T13:39:39.441996+0000","last_unstale":"2024-08-18T13:52:35.155057+0000","last_undegraded":"2024-08-18T13:52:35.155057+0000","last_fullsized":"2024-08-18T13:52:35.155057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'661","last_scrub_stamp":"2024-08-18T13:52:35.154884+0000","last_deep_scrub":"54'661","last_deep_scrub_stamp":"2024-08-18T13:52:35.154884+0000","last_clean_scrub_stamp":"2024-08-18T13:52:35.154884+0000","objects_scrubbed":345,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:59:12.312487+0000","scrub_duration":0.140792578,"objects_trimmed":0,"snaptrim_duration":0.0067969390000000001,"stat_sum":{"num_bytes":51061167,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":1737,"num_read_kb":43040,"num_write":661,"num_write_kb":53481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"54'83","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.003797+0000","last_change":"2024-08-18T13:52:18.003732+0000","last_active":"2024-08-18T13:52:18.003797+0000","last_peered":"2024-08-18T13:52:18.003797+0000","last_clean":"2024-08-18T13:52:18.003797+0000","last_became_active":"2024-08-18T13:39:35.400108+0000","last_became_peered":"2024-08-18T13:39:35.400108+0000","last_unstale":"2024-08-18T13:52:18.003797+0000","last_undegraded":"2024-08-18T13:52:18.003797+0000","last_fullsized":"2024-08-18T13:52:18.003797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'83","last_scrub_stamp":"2024-08-18T13:52:18.003500+0000","last_deep_scrub":"54'83","last_deep_scrub_stamp":"2024-08-18T13:52:18.003500+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.003500+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:20.942863+0000","scrub_duration":0.0033910189999999999,"objects_trimmed":0,"snaptrim_duration":0.0063915389999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'122","reported_seq":296,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:11.436136+0000","last_change":"2024-08-18T13:52:11.436080+0000","last_active":"2024-08-18T13:52:11.436136+0000","last_peered":"2024-08-18T13:52:11.436136+0000","last_clean":"2024-08-18T13:52:11.436136+0000","last_became_active":"2024-08-18T13:39:34.381843+0000","last_became_peered":"2024-08-18T13:39:34.381843+0000","last_unstale":"2024-08-18T13:52:11.436136+0000","last_undegraded":"2024-08-18T13:52:11.436136+0000","last_fullsized":"2024-08-18T13:52:11.436136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'122","last_scrub_stamp":"2024-08-18T13:52:11.435881+0000","last_deep_scrub":"54'122","last_deep_scrub_stamp":"2024-08-18T13:52:11.435881+0000","last_clean_scrub_stamp":"2024-08-18T13:52:11.435881+0000","objects_scrubbed":2,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:53:26.277110+0000","scrub_duration":0.010000867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":575719,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":638,"num_write":117,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14952,"num_omap_keys":28,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"54'643","reported_seq":3620,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:39.154461+0000","last_change":"2024-08-18T13:52:39.154377+0000","last_active":"2024-08-18T13:52:39.154461+0000","last_peered":"2024-08-18T13:52:39.154461+0000","last_clean":"2024-08-18T13:52:39.154461+0000","last_became_active":"2024-08-18T13:39:39.445990+0000","last_became_peered":"2024-08-18T13:39:39.445990+0000","last_unstale":"2024-08-18T13:52:39.154461+0000","last_undegraded":"2024-08-18T13:52:39.154461+0000","last_fullsized":"2024-08-18T13:52:39.154461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'643","last_scrub_stamp":"2024-08-18T13:52:39.154224+0000","last_deep_scrub":"54'643","last_deep_scrub_stamp":"2024-08-18T13:52:39.154224+0000","last_clean_scrub_stamp":"2024-08-18T13:52:39.154224+0000","objects_scrubbed":307,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:56:41.728995+0000","scrub_duration":0.12558661199999999,"objects_trimmed":0,"snaptrim_duration":0.0079242040000000007,"stat_sum":{"num_bytes":41790406,"num_objects":307,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":307,"num_whiteouts":0,"num_read":892,"num_read_kb":23319,"num_write":639,"num_write_kb":45248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'82","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:12.049564+0000","last_change":"2024-08-18T13:52:12.049490+0000","last_active":"2024-08-18T13:52:12.049564+0000","last_peered":"2024-08-18T13:52:12.049564+0000","last_clean":"2024-08-18T13:52:12.049564+0000","last_became_active":"2024-08-18T13:39:35.408692+0000","last_became_peered":"2024-08-18T13:39:35.408692+0000","last_unstale":"2024-08-18T13:52:12.049564+0000","last_undegraded":"2024-08-18T13:52:12.049564+0000","last_fullsized":"2024-08-18T13:52:12.049564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'82","last_scrub_stamp":"2024-08-18T13:52:12.049277+0000","last_deep_scrub":"54'82","last_deep_scrub_stamp":"2024-08-18T13:52:12.049277+0000","last_clean_scrub_stamp":"2024-08-18T13:52:12.049277+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:42:58.521893+0000","scrub_duration":0.0038715189999999999,"objects_trimmed":0,"snaptrim_duration":0.0041495459999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"54'240","reported_seq":307,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:13.957043+0000","last_change":"2024-08-18T13:52:13.956993+0000","last_active":"2024-08-18T13:52:13.957043+0000","last_peered":"2024-08-18T13:52:13.957043+0000","last_clean":"2024-08-18T13:52:13.957043+0000","last_became_active":"2024-08-18T13:39:34.395000+0000","last_became_peered":"2024-08-18T13:39:34.395000+0000","last_unstale":"2024-08-18T13:52:13.957043+0000","last_undegraded":"2024-08-18T13:52:13.957043+0000","last_fullsized":"2024-08-18T13:52:13.957043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'240","last_scrub_stamp":"2024-08-18T13:52:13.956898+0000","last_deep_scrub":"54'240","last_deep_scrub_stamp":"2024-08-18T13:52:13.956898+0000","last_clean_scrub_stamp":"2024-08-18T13:52:13.956898+0000","objects_scrubbed":6,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:30:29.246162+0000","scrub_duration":0.0255698,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":49152,"num_write":231,"num_write_kb":12421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24290,"num_omap_keys":46,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"54'716","reported_seq":3241,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:32.135169+0000","last_change":"2024-08-18T13:52:32.135117+0000","last_active":"2024-08-18T13:52:32.135169+0000","last_peered":"2024-08-18T13:52:32.135169+0000","last_clean":"2024-08-18T13:52:32.135169+0000","last_became_active":"2024-08-18T13:39:39.453143+0000","last_became_peered":"2024-08-18T13:39:39.453143+0000","last_unstale":"2024-08-18T13:52:32.135169+0000","last_undegraded":"2024-08-18T13:52:32.135169+0000","last_fullsized":"2024-08-18T13:52:32.135169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'716","last_scrub_stamp":"2024-08-18T13:52:32.134994+0000","last_deep_scrub":"54'716","last_deep_scrub_stamp":"2024-08-18T13:52:32.134994+0000","last_clean_scrub_stamp":"2024-08-18T13:52:32.134994+0000","objects_scrubbed":352,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:10.068446+0000","scrub_duration":0.13071303000000001,"objects_trimmed":0,"snaptrim_duration":0.0058974379999999996,"stat_sum":{"num_bytes":30462084,"num_objects":352,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":439,"num_read_kb":15189,"num_write":710,"num_write_kb":30067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"54'90","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.918324+0000","last_change":"2024-08-18T13:52:09.918219+0000","last_active":"2024-08-18T13:52:09.918324+0000","last_peered":"2024-08-18T13:52:09.918324+0000","last_clean":"2024-08-18T13:52:09.918324+0000","last_became_active":"2024-08-18T13:39:35.411821+0000","last_became_peered":"2024-08-18T13:39:35.411821+0000","last_unstale":"2024-08-18T13:52:09.918324+0000","last_undegraded":"2024-08-18T13:52:09.918324+0000","last_fullsized":"2024-08-18T13:52:09.918324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:09.917892+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:09.917892+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.917892+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:20.972039+0000","scrub_duration":0.0039382109999999996,"objects_trimmed":0,"snaptrim_duration":0.0068182370000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"54'983","reported_seq":1039,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.478599+0000","last_change":"2024-08-18T13:52:07.478553+0000","last_active":"2024-08-18T13:52:07.478599+0000","last_peered":"2024-08-18T13:52:07.478599+0000","last_clean":"2024-08-18T13:52:07.478599+0000","last_became_active":"2024-08-18T13:39:34.382864+0000","last_became_peered":"2024-08-18T13:39:34.382864+0000","last_unstale":"2024-08-18T13:52:07.478599+0000","last_undegraded":"2024-08-18T13:52:07.478599+0000","last_fullsized":"2024-08-18T13:52:07.478599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'983","last_scrub_stamp":"2024-08-18T13:52:07.478392+0000","last_deep_scrub":"54'983","last_deep_scrub_stamp":"2024-08-18T13:52:07.478392+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.478392+0000","objects_scrubbed":2,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:52:04.274947+0000","scrub_duration":0.015274658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2087,"num_write":979,"num_write_kb":4688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8826,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"54'119","reported_seq":208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:06.906625+0000","last_change":"2024-08-18T13:52:06.906561+0000","last_active":"2024-08-18T13:52:06.906625+0000","last_peered":"2024-08-18T13:52:06.906625+0000","last_clean":"2024-08-18T13:52:06.906625+0000","last_became_active":"2024-08-18T13:39:22.780766+0000","last_became_peered":"2024-08-18T13:39:22.780766+0000","last_unstale":"2024-08-18T13:52:06.906625+0000","last_undegraded":"2024-08-18T13:52:06.906625+0000","last_fullsized":"2024-08-18T13:52:06.906625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'119","last_scrub_stamp":"2024-08-18T13:52:06.906373+0000","last_deep_scrub":"54'119","last_deep_scrub_stamp":"2024-08-18T13:52:06.906373+0000","last_clean_scrub_stamp":"2024-08-18T13:52:06.906373+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:57:50.221813+0000","scrub_duration":0.0082944209999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"54'664","reported_seq":3549,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:32.413005+0000","last_change":"2024-08-18T13:52:32.412930+0000","last_active":"2024-08-18T13:52:32.413005+0000","last_peered":"2024-08-18T13:52:32.413005+0000","last_clean":"2024-08-18T13:52:32.413005+0000","last_became_active":"2024-08-18T13:39:39.452857+0000","last_became_peered":"2024-08-18T13:39:39.452857+0000","last_unstale":"2024-08-18T13:52:32.413005+0000","last_undegraded":"2024-08-18T13:52:32.413005+0000","last_fullsized":"2024-08-18T13:52:32.413005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'664","last_scrub_stamp":"2024-08-18T13:52:32.412772+0000","last_deep_scrub":"54'664","last_deep_scrub_stamp":"2024-08-18T13:52:32.412772+0000","last_clean_scrub_stamp":"2024-08-18T13:52:32.412772+0000","objects_scrubbed":364,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:29:53.313639+0000","scrub_duration":0.15394601399999999,"objects_trimmed":0,"snaptrim_duration":0.0035684480000000001,"stat_sum":{"num_bytes":39930170,"num_objects":364,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":798,"num_read_kb":26403,"num_write":662,"num_write_kb":47533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:24.899358+0000","last_change":"2024-08-18T13:52:24.899284+0000","last_active":"2024-08-18T13:52:24.899358+0000","last_peered":"2024-08-18T13:52:24.899358+0000","last_clean":"2024-08-18T13:52:24.899358+0000","last_became_active":"2024-08-18T13:39:35.421304+0000","last_became_peered":"2024-08-18T13:39:35.421304+0000","last_unstale":"2024-08-18T13:52:24.899358+0000","last_undegraded":"2024-08-18T13:52:24.899358+0000","last_fullsized":"2024-08-18T13:52:24.899358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:24.899079+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:24.899079+0000","last_clean_scrub_stamp":"2024-08-18T13:52:24.899079+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:46:27.076634+0000","scrub_duration":0.0037898570000000002,"objects_trimmed":0,"snaptrim_duration":0.0057711639999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"54'80","reported_seq":133,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.495618+0000","last_change":"2024-08-18T13:52:16.495514+0000","last_active":"2024-08-18T13:52:16.495618+0000","last_peered":"2024-08-18T13:52:16.495618+0000","last_clean":"2024-08-18T13:52:16.495618+0000","last_became_active":"2024-08-18T13:39:34.382542+0000","last_became_peered":"2024-08-18T13:39:34.382542+0000","last_unstale":"2024-08-18T13:52:16.495618+0000","last_undegraded":"2024-08-18T13:52:16.495618+0000","last_fullsized":"2024-08-18T13:52:16.495618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'80","last_scrub_stamp":"2024-08-18T13:52:16.495259+0000","last_deep_scrub":"54'80","last_deep_scrub_stamp":"2024-08-18T13:52:16.495259+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.495259+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:47:37.229120+0000","scrub_duration":0.016550458000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31498,"num_omap_keys":60,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"54'688","reported_seq":4151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.418732+0000","last_change":"2024-08-18T13:52:31.418655+0000","last_active":"2024-08-18T13:52:31.418732+0000","last_peered":"2024-08-18T13:52:31.418732+0000","last_clean":"2024-08-18T13:52:31.418732+0000","last_became_active":"2024-08-18T13:39:39.448868+0000","last_became_peered":"2024-08-18T13:39:39.448868+0000","last_unstale":"2024-08-18T13:52:31.418732+0000","last_undegraded":"2024-08-18T13:52:31.418732+0000","last_fullsized":"2024-08-18T13:52:31.418732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'688","last_scrub_stamp":"2024-08-18T13:52:31.418510+0000","last_deep_scrub":"54'688","last_deep_scrub_stamp":"2024-08-18T13:52:31.418510+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.418510+0000","objects_scrubbed":349,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:54:04.889299+0000","scrub_duration":0.17638959800000001,"objects_trimmed":0,"snaptrim_duration":0.0032433420000000002,"stat_sum":{"num_bytes":54557211,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1377,"num_read_kb":50055,"num_write":687,"num_write_kb":57734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"54'95","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.530780+0000","last_change":"2024-08-18T13:52:27.530717+0000","last_active":"2024-08-18T13:52:27.530780+0000","last_peered":"2024-08-18T13:52:27.530780+0000","last_clean":"2024-08-18T13:52:27.530780+0000","last_became_active":"2024-08-18T13:40:16.614814+0000","last_became_peered":"2024-08-18T13:40:16.614814+0000","last_unstale":"2024-08-18T13:52:27.530780+0000","last_undegraded":"2024-08-18T13:52:27.530780+0000","last_fullsized":"2024-08-18T13:52:27.530780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'95","last_scrub_stamp":"2024-08-18T13:52:27.530504+0000","last_deep_scrub":"54'95","last_deep_scrub_stamp":"2024-08-18T13:52:27.530504+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.530504+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:13.569364+0000","scrub_duration":0.003176417,"objects_trimmed":0,"snaptrim_duration":0.0033975450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'36","reported_seq":101,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.170255+0000","last_change":"2024-08-18T13:52:10.170197+0000","last_active":"2024-08-18T13:52:10.170255+0000","last_peered":"2024-08-18T13:52:10.170255+0000","last_clean":"2024-08-18T13:52:10.170255+0000","last_became_active":"2024-08-18T13:39:34.384406+0000","last_became_peered":"2024-08-18T13:39:34.384406+0000","last_unstale":"2024-08-18T13:52:10.170255+0000","last_undegraded":"2024-08-18T13:52:10.170255+0000","last_fullsized":"2024-08-18T13:52:10.170255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'36","last_scrub_stamp":"2024-08-18T13:52:10.170013+0000","last_deep_scrub":"54'36","last_deep_scrub_stamp":"2024-08-18T13:52:10.170013+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.170013+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:39:21.224160+0000","scrub_duration":0.017405436999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":33,"num_write_kb":4137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24938,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"54'670","reported_seq":3259,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:43.266554+0000","last_change":"2024-08-18T13:52:43.266386+0000","last_active":"2024-08-18T13:52:43.266554+0000","last_peered":"2024-08-18T13:52:43.266554+0000","last_clean":"2024-08-18T13:52:43.266554+0000","last_became_active":"2024-08-18T13:40:16.609908+0000","last_became_peered":"2024-08-18T13:40:16.609908+0000","last_unstale":"2024-08-18T13:52:43.266554+0000","last_undegraded":"2024-08-18T13:52:43.266554+0000","last_fullsized":"2024-08-18T13:52:43.266554+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'670","last_scrub_stamp":"2024-08-18T13:52:43.266185+0000","last_deep_scrub":"54'670","last_deep_scrub_stamp":"2024-08-18T13:52:43.266185+0000","last_clean_scrub_stamp":"2024-08-18T13:52:43.266185+0000","objects_scrubbed":358,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:56:45.840956+0000","scrub_duration":0.13026812300000001,"objects_trimmed":0,"snaptrim_duration":0.0069871079999999997,"stat_sum":{"num_bytes":26719276,"num_objects":358,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":491,"num_read_kb":18148,"num_write":671,"num_write_kb":26414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4,5],"acting":[0,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"54'87","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:32.867441+0000","last_change":"2024-08-18T13:52:32.867312+0000","last_active":"2024-08-18T13:52:32.867441+0000","last_peered":"2024-08-18T13:52:32.867441+0000","last_clean":"2024-08-18T13:52:32.867441+0000","last_became_active":"2024-08-18T13:39:35.420898+0000","last_became_peered":"2024-08-18T13:39:35.420898+0000","last_unstale":"2024-08-18T13:52:32.867441+0000","last_undegraded":"2024-08-18T13:52:32.867441+0000","last_fullsized":"2024-08-18T13:52:32.867441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'87","last_scrub_stamp":"2024-08-18T13:52:32.867113+0000","last_deep_scrub":"54'87","last_deep_scrub_stamp":"2024-08-18T13:52:32.867113+0000","last_clean_scrub_stamp":"2024-08-18T13:52:32.867113+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:29:53.767979+0000","scrub_duration":0.003763452,"objects_trimmed":0,"snaptrim_duration":0.0057951929999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"54'32","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.525558+0000","last_change":"2024-08-18T13:52:18.525490+0000","last_active":"2024-08-18T13:52:18.525558+0000","last_peered":"2024-08-18T13:52:18.525558+0000","last_clean":"2024-08-18T13:52:18.525558+0000","last_became_active":"2024-08-18T13:39:34.386877+0000","last_became_peered":"2024-08-18T13:39:34.386877+0000","last_unstale":"2024-08-18T13:52:18.525558+0000","last_undegraded":"2024-08-18T13:52:18.525558+0000","last_fullsized":"2024-08-18T13:52:18.525558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'32","last_scrub_stamp":"2024-08-18T13:52:18.525224+0000","last_deep_scrub":"54'32","last_deep_scrub_stamp":"2024-08-18T13:52:18.525224+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.525224+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:52:15.321780+0000","scrub_duration":0.016102814999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":20498,"num_write":30,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9612,"num_omap_keys":18,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"54'688","reported_seq":2842,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.583212+0000","last_change":"2024-08-18T13:52:34.583104+0000","last_active":"2024-08-18T13:52:34.583212+0000","last_peered":"2024-08-18T13:52:34.583212+0000","last_clean":"2024-08-18T13:52:34.583212+0000","last_became_active":"2024-08-18T13:39:39.453034+0000","last_became_peered":"2024-08-18T13:39:39.453034+0000","last_unstale":"2024-08-18T13:52:34.583212+0000","last_undegraded":"2024-08-18T13:52:34.583212+0000","last_fullsized":"2024-08-18T13:52:34.583212+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'688","last_scrub_stamp":"2024-08-18T13:52:34.582985+0000","last_deep_scrub":"54'688","last_deep_scrub_stamp":"2024-08-18T13:52:34.582985+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.582985+0000","objects_scrubbed":346,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:53:19.774572+0000","scrub_duration":0.112945194,"objects_trimmed":0,"snaptrim_duration":0.0070733389999999997,"stat_sum":{"num_bytes":17282460,"num_objects":346,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":68,"num_read_kb":8224,"num_write":686,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"54'66","reported_seq":2140,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.910326+0000","last_change":"2024-08-18T13:52:04.910259+0000","last_active":"2024-08-18T13:52:04.910326+0000","last_peered":"2024-08-18T13:52:04.910326+0000","last_clean":"2024-08-18T13:52:04.910326+0000","last_became_active":"2024-08-18T13:39:35.421142+0000","last_became_peered":"2024-08-18T13:39:35.421142+0000","last_unstale":"2024-08-18T13:52:04.910326+0000","last_undegraded":"2024-08-18T13:52:04.910326+0000","last_fullsized":"2024-08-18T13:52:04.910326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'66","last_scrub_stamp":"2024-08-18T13:52:04.910054+0000","last_deep_scrub":"54'66","last_deep_scrub_stamp":"2024-08-18T13:52:04.910054+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.910054+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:14:14.576424+0000","scrub_duration":0.0032501320000000002,"objects_trimmed":0,"snaptrim_duration":0.0060906930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"54'29","reported_seq":94,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.600601+0000","last_change":"2024-08-18T13:52:04.600549+0000","last_active":"2024-08-18T13:52:04.600601+0000","last_peered":"2024-08-18T13:52:04.600601+0000","last_clean":"2024-08-18T13:52:04.600601+0000","last_became_active":"2024-08-18T13:39:34.387405+0000","last_became_peered":"2024-08-18T13:39:34.387405+0000","last_unstale":"2024-08-18T13:52:04.600601+0000","last_undegraded":"2024-08-18T13:52:04.600601+0000","last_fullsized":"2024-08-18T13:52:04.600601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'29","last_scrub_stamp":"2024-08-18T13:52:04.600439+0000","last_deep_scrub":"54'29","last_deep_scrub_stamp":"2024-08-18T13:52:04.600439+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.600439+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:26.017443+0000","scrub_duration":0.014861572999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":20486,"num_write":27,"num_write_kb":4177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"54'695","reported_seq":3654,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:33.838023+0000","last_change":"2024-08-18T13:52:33.837968+0000","last_active":"2024-08-18T13:52:33.838023+0000","last_peered":"2024-08-18T13:52:33.838023+0000","last_clean":"2024-08-18T13:52:33.838023+0000","last_became_active":"2024-08-18T13:39:39.444782+0000","last_became_peered":"2024-08-18T13:39:39.444782+0000","last_unstale":"2024-08-18T13:52:33.838023+0000","last_undegraded":"2024-08-18T13:52:33.838023+0000","last_fullsized":"2024-08-18T13:52:33.838023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'695","last_scrub_stamp":"2024-08-18T13:52:33.837813+0000","last_deep_scrub":"54'695","last_deep_scrub_stamp":"2024-08-18T13:52:33.837813+0000","last_clean_scrub_stamp":"2024-08-18T13:52:33.837813+0000","objects_scrubbed":345,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:56:26.045133+0000","scrub_duration":0.159429342,"objects_trimmed":0,"snaptrim_duration":0.011606655,"stat_sum":{"num_bytes":44119789,"num_objects":345,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":873,"num_read_kb":37157,"num_write":696,"num_write_kb":47501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"54'91","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.858350+0000","last_change":"2024-08-18T13:52:30.858279+0000","last_active":"2024-08-18T13:52:30.858350+0000","last_peered":"2024-08-18T13:52:30.858350+0000","last_clean":"2024-08-18T13:52:30.858350+0000","last_became_active":"2024-08-18T13:39:35.414909+0000","last_became_peered":"2024-08-18T13:39:35.414909+0000","last_unstale":"2024-08-18T13:52:30.858350+0000","last_undegraded":"2024-08-18T13:52:30.858350+0000","last_fullsized":"2024-08-18T13:52:30.858350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:30.858089+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:30.858089+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.858089+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:32:05.996511+0000","scrub_duration":0.0036829689999999999,"objects_trimmed":0,"snaptrim_duration":0.0053019490000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"54'1212","reported_seq":1295,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.493306+0000","last_change":"2024-08-18T13:52:14.493275+0000","last_active":"2024-08-18T13:52:14.493306+0000","last_peered":"2024-08-18T13:52:14.493306+0000","last_clean":"2024-08-18T13:52:14.493306+0000","last_became_active":"2024-08-18T13:39:34.378139+0000","last_became_peered":"2024-08-18T13:39:34.378139+0000","last_unstale":"2024-08-18T13:52:14.493306+0000","last_undegraded":"2024-08-18T13:52:14.493306+0000","last_fullsized":"2024-08-18T13:52:14.493306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'1212","last_scrub_stamp":"2024-08-18T13:52:14.493191+0000","last_deep_scrub":"54'1212","last_deep_scrub_stamp":"2024-08-18T13:52:14.493191+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.493191+0000","objects_scrubbed":8,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:29:04.116014+0000","scrub_duration":0.026335695999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":78,"num_read_kb":12399,"num_write":1202,"num_write_kb":8830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18540,"num_omap_keys":36,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'657","reported_seq":3538,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:33.123467+0000","last_change":"2024-08-18T13:52:33.123383+0000","last_active":"2024-08-18T13:52:33.123467+0000","last_peered":"2024-08-18T13:52:33.123467+0000","last_clean":"2024-08-18T13:52:33.123467+0000","last_became_active":"2024-08-18T13:39:39.436562+0000","last_became_peered":"2024-08-18T13:39:39.436562+0000","last_unstale":"2024-08-18T13:52:33.123467+0000","last_undegraded":"2024-08-18T13:52:33.123467+0000","last_fullsized":"2024-08-18T13:52:33.123467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'657","last_scrub_stamp":"2024-08-18T13:52:33.123217+0000","last_deep_scrub":"54'657","last_deep_scrub_stamp":"2024-08-18T13:52:33.123217+0000","last_clean_scrub_stamp":"2024-08-18T13:52:33.123217+0000","objects_scrubbed":296,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:08.246627+0000","scrub_duration":0.120851848,"objects_trimmed":0,"snaptrim_duration":0.0054104280000000001,"stat_sum":{"num_bytes":33280018,"num_objects":296,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":296,"num_whiteouts":0,"num_read":797,"num_read_kb":22024,"num_write":655,"num_write_kb":32783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.315208+0000","last_change":"2024-08-18T13:52:18.315124+0000","last_active":"2024-08-18T13:52:18.315208+0000","last_peered":"2024-08-18T13:52:18.315208+0000","last_clean":"2024-08-18T13:52:18.315208+0000","last_became_active":"2024-08-18T13:39:35.410981+0000","last_became_peered":"2024-08-18T13:39:35.410981+0000","last_unstale":"2024-08-18T13:52:18.315208+0000","last_undegraded":"2024-08-18T13:52:18.315208+0000","last_fullsized":"2024-08-18T13:52:18.315208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:18.314906+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:18.314906+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.314906+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:39.731910+0000","scrub_duration":0.0038296580000000001,"objects_trimmed":0,"snaptrim_duration":0.0044255090000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"54'8","reported_seq":61,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:09.301096+0000","last_change":"2024-08-18T13:52:09.301035+0000","last_active":"2024-08-18T13:52:09.301096+0000","last_peered":"2024-08-18T13:52:09.301096+0000","last_clean":"2024-08-18T13:52:09.301096+0000","last_became_active":"2024-08-18T13:39:34.384925+0000","last_became_peered":"2024-08-18T13:39:34.384925+0000","last_unstale":"2024-08-18T13:52:09.301096+0000","last_undegraded":"2024-08-18T13:52:09.301096+0000","last_fullsized":"2024-08-18T13:52:09.301096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'8","last_scrub_stamp":"2024-08-18T13:52:09.300822+0000","last_deep_scrub":"54'8","last_deep_scrub_stamp":"2024-08-18T13:52:09.300822+0000","last_clean_scrub_stamp":"2024-08-18T13:52:09.300822+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:39:45.416911+0000","scrub_duration":0.008909953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28532,"num_omap_keys":54,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"54'660","reported_seq":2804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:44.456541+0000","last_change":"2024-08-18T13:52:44.456425+0000","last_active":"2024-08-18T13:52:44.456541+0000","last_peered":"2024-08-18T13:52:44.456541+0000","last_clean":"2024-08-18T13:52:44.456541+0000","last_became_active":"2024-08-18T13:40:16.915369+0000","last_became_peered":"2024-08-18T13:40:16.915369+0000","last_unstale":"2024-08-18T13:52:44.456541+0000","last_undegraded":"2024-08-18T13:52:44.456541+0000","last_fullsized":"2024-08-18T13:52:44.456541+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'660","last_scrub_stamp":"2024-08-18T13:52:44.456224+0000","last_deep_scrub":"54'660","last_deep_scrub_stamp":"2024-08-18T13:52:44.456224+0000","last_clean_scrub_stamp":"2024-08-18T13:52:44.456224+0000","objects_scrubbed":353,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:02:20.711412+0000","scrub_duration":0.15007590800000001,"objects_trimmed":0,"snaptrim_duration":0.005619059,"stat_sum":{"num_bytes":16977484,"num_objects":353,"num_object_clones":0,"num_object_copies":1412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":46,"num_read_kb":4277,"num_write":660,"num_write_kb":20975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,6],"acting":[3,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"54'101","reported_seq":2175,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:25.870786+0000","last_change":"2024-08-18T13:52:25.870697+0000","last_active":"2024-08-18T13:52:25.870786+0000","last_peered":"2024-08-18T13:52:25.870786+0000","last_clean":"2024-08-18T13:52:25.870786+0000","last_became_active":"2024-08-18T13:39:35.417065+0000","last_became_peered":"2024-08-18T13:39:35.417065+0000","last_unstale":"2024-08-18T13:52:25.870786+0000","last_undegraded":"2024-08-18T13:52:25.870786+0000","last_fullsized":"2024-08-18T13:52:25.870786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'101","last_scrub_stamp":"2024-08-18T13:52:25.870489+0000","last_deep_scrub":"54'101","last_deep_scrub_stamp":"2024-08-18T13:52:25.870489+0000","last_clean_scrub_stamp":"2024-08-18T13:52:25.870489+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:19:17.425779+0000","scrub_duration":0.0032600789999999999,"objects_trimmed":0,"snaptrim_duration":0.0064019180000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"54'4","reported_seq":58,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.395883+0000","last_change":"2024-08-18T13:52:15.395814+0000","last_active":"2024-08-18T13:52:15.395883+0000","last_peered":"2024-08-18T13:52:15.395883+0000","last_clean":"2024-08-18T13:52:15.395883+0000","last_became_active":"2024-08-18T13:39:34.393693+0000","last_became_peered":"2024-08-18T13:39:34.393693+0000","last_unstale":"2024-08-18T13:52:15.395883+0000","last_undegraded":"2024-08-18T13:52:15.395883+0000","last_fullsized":"2024-08-18T13:52:15.395883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'4","last_scrub_stamp":"2024-08-18T13:52:15.395609+0000","last_deep_scrub":"54'4","last_deep_scrub_stamp":"2024-08-18T13:52:15.395609+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.395609+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:46.408855+0000","scrub_duration":0.0073810990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15112,"num_omap_keys":29,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"54'625","reported_seq":3622,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:41.396892+0000","last_change":"2024-08-18T13:52:41.396824+0000","last_active":"2024-08-18T13:52:41.396892+0000","last_peered":"2024-08-18T13:52:41.396892+0000","last_clean":"2024-08-18T13:52:41.396892+0000","last_became_active":"2024-08-18T13:39:39.453259+0000","last_became_peered":"2024-08-18T13:39:39.453259+0000","last_unstale":"2024-08-18T13:52:41.396892+0000","last_undegraded":"2024-08-18T13:52:41.396892+0000","last_fullsized":"2024-08-18T13:52:41.396892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'625","last_scrub_stamp":"2024-08-18T13:52:41.396659+0000","last_deep_scrub":"54'625","last_deep_scrub_stamp":"2024-08-18T13:52:41.396659+0000","last_clean_scrub_stamp":"2024-08-18T13:52:41.396659+0000","objects_scrubbed":332,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:12:11.204418+0000","scrub_duration":0.135165489,"objects_trimmed":0,"snaptrim_duration":0.0074125900000000002,"stat_sum":{"num_bytes":40693231,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":911,"num_read_kb":37155,"num_write":622,"num_write_kb":48255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:08.056426+0000","last_change":"2024-08-18T13:52:08.056345+0000","last_active":"2024-08-18T13:52:08.056426+0000","last_peered":"2024-08-18T13:52:08.056426+0000","last_clean":"2024-08-18T13:52:08.056426+0000","last_became_active":"2024-08-18T13:39:35.404140+0000","last_became_peered":"2024-08-18T13:39:35.404140+0000","last_unstale":"2024-08-18T13:52:08.056426+0000","last_undegraded":"2024-08-18T13:52:08.056426+0000","last_fullsized":"2024-08-18T13:52:08.056426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:08.056130+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:08.056130+0000","last_clean_scrub_stamp":"2024-08-18T13:52:08.056130+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:54.094989+0000","scrub_duration":0.003803248,"objects_trimmed":0,"snaptrim_duration":0.0059765870000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"54'220","reported_seq":298,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:02.096320+0000","last_change":"2024-08-18T13:52:02.096276+0000","last_active":"2024-08-18T13:52:02.096320+0000","last_peered":"2024-08-18T13:52:02.096320+0000","last_clean":"2024-08-18T13:52:02.096320+0000","last_became_active":"2024-08-18T13:39:34.378271+0000","last_became_peered":"2024-08-18T13:39:34.378271+0000","last_unstale":"2024-08-18T13:52:02.096320+0000","last_undegraded":"2024-08-18T13:52:02.096320+0000","last_fullsized":"2024-08-18T13:52:02.096320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'220","last_scrub_stamp":"2024-08-18T13:52:02.096123+0000","last_deep_scrub":"54'220","last_deep_scrub_stamp":"2024-08-18T13:52:02.096123+0000","last_clean_scrub_stamp":"2024-08-18T13:52:02.096123+0000","objects_scrubbed":6,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:09.274537+0000","scrub_duration":0.023746768000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":45098,"num_write":215,"num_write_kb":8331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11822,"num_omap_keys":23,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'629","reported_seq":3964,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:39.478236+0000","last_change":"2024-08-18T13:52:39.478158+0000","last_active":"2024-08-18T13:52:39.478236+0000","last_peered":"2024-08-18T13:52:39.478236+0000","last_clean":"2024-08-18T13:52:39.478236+0000","last_became_active":"2024-08-18T13:39:39.456039+0000","last_became_peered":"2024-08-18T13:39:39.456039+0000","last_unstale":"2024-08-18T13:52:39.478236+0000","last_undegraded":"2024-08-18T13:52:39.478236+0000","last_fullsized":"2024-08-18T13:52:39.478236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'629","last_scrub_stamp":"2024-08-18T13:52:39.478005+0000","last_deep_scrub":"54'629","last_deep_scrub_stamp":"2024-08-18T13:52:39.478005+0000","last_clean_scrub_stamp":"2024-08-18T13:52:39.478005+0000","objects_scrubbed":306,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:48.229574+0000","scrub_duration":0.147969562,"objects_trimmed":0,"snaptrim_duration":0.0084356780000000003,"stat_sum":{"num_bytes":50488256,"num_objects":306,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":1250,"num_read_kb":43248,"num_write":629,"num_write_kb":49603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"54'84","reported_seq":2158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:26.838724+0000","last_change":"2024-08-18T13:52:26.838660+0000","last_active":"2024-08-18T13:52:26.838724+0000","last_peered":"2024-08-18T13:52:26.838724+0000","last_clean":"2024-08-18T13:52:26.838724+0000","last_became_active":"2024-08-18T13:39:35.417378+0000","last_became_peered":"2024-08-18T13:39:35.417378+0000","last_unstale":"2024-08-18T13:52:26.838724+0000","last_undegraded":"2024-08-18T13:52:26.838724+0000","last_fullsized":"2024-08-18T13:52:26.838724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'84","last_scrub_stamp":"2024-08-18T13:52:26.838465+0000","last_deep_scrub":"54'84","last_deep_scrub_stamp":"2024-08-18T13:52:26.838465+0000","last_clean_scrub_stamp":"2024-08-18T13:52:26.838465+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:59:19.259738+0000","scrub_duration":0.003882019,"objects_trimmed":0,"snaptrim_duration":0.0065977270000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"54'241","reported_seq":325,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:12.244378+0000","last_change":"2024-08-18T13:52:12.244303+0000","last_active":"2024-08-18T13:52:12.244378+0000","last_peered":"2024-08-18T13:52:12.244378+0000","last_clean":"2024-08-18T13:52:12.244378+0000","last_became_active":"2024-08-18T13:39:34.394548+0000","last_became_peered":"2024-08-18T13:39:34.394548+0000","last_unstale":"2024-08-18T13:52:12.244378+0000","last_undegraded":"2024-08-18T13:52:12.244378+0000","last_fullsized":"2024-08-18T13:52:12.244378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'240","last_scrub_stamp":"2024-08-18T13:52:12.244185+0000","last_deep_scrub":"54'240","last_deep_scrub_stamp":"2024-08-18T13:52:12.244185+0000","last_clean_scrub_stamp":"2024-08-18T13:52:12.244185+0000","objects_scrubbed":8,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:30:27.533449+0000","scrub_duration":0.029778697999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583572,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":53278,"num_write":227,"num_write_kb":12433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22962,"num_omap_keys":43,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"54'671","reported_seq":3286,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.098343+0000","last_change":"2024-08-18T13:52:30.098271+0000","last_active":"2024-08-18T13:52:30.098343+0000","last_peered":"2024-08-18T13:52:30.098343+0000","last_clean":"2024-08-18T13:52:30.098343+0000","last_became_active":"2024-08-18T13:39:39.437655+0000","last_became_peered":"2024-08-18T13:39:39.437655+0000","last_unstale":"2024-08-18T13:52:30.098343+0000","last_undegraded":"2024-08-18T13:52:30.098343+0000","last_fullsized":"2024-08-18T13:52:30.098343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'671","last_scrub_stamp":"2024-08-18T13:52:30.098121+0000","last_deep_scrub":"54'671","last_deep_scrub_stamp":"2024-08-18T13:52:30.098121+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.098121+0000","objects_scrubbed":343,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:05.221530+0000","scrub_duration":0.130780905,"objects_trimmed":0,"snaptrim_duration":0.0076714670000000004,"stat_sum":{"num_bytes":16086174,"num_objects":343,"num_object_clones":0,"num_object_copies":1372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":529,"num_read_kb":12373,"num_write":668,"num_write_kb":24206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"54'86","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:15.670616+0000","last_change":"2024-08-18T13:52:15.670550+0000","last_active":"2024-08-18T13:52:15.670616+0000","last_peered":"2024-08-18T13:52:15.670616+0000","last_clean":"2024-08-18T13:52:15.670616+0000","last_became_active":"2024-08-18T13:39:35.405930+0000","last_became_peered":"2024-08-18T13:39:35.405930+0000","last_unstale":"2024-08-18T13:52:15.670616+0000","last_undegraded":"2024-08-18T13:52:15.670616+0000","last_fullsized":"2024-08-18T13:52:15.670616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'86","last_scrub_stamp":"2024-08-18T13:52:15.670343+0000","last_deep_scrub":"54'86","last_deep_scrub_stamp":"2024-08-18T13:52:15.670343+0000","last_clean_scrub_stamp":"2024-08-18T13:52:15.670343+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:15:49.587682+0000","scrub_duration":0.0034242740000000002,"objects_trimmed":0,"snaptrim_duration":0.0039212459999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"54'173","reported_seq":242,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.045210+0000","last_change":"2024-08-18T13:52:14.045155+0000","last_active":"2024-08-18T13:52:14.045210+0000","last_peered":"2024-08-18T13:52:14.045210+0000","last_clean":"2024-08-18T13:52:14.045210+0000","last_became_active":"2024-08-18T13:39:34.385830+0000","last_became_peered":"2024-08-18T13:39:34.385830+0000","last_unstale":"2024-08-18T13:52:14.045210+0000","last_undegraded":"2024-08-18T13:52:14.045210+0000","last_fullsized":"2024-08-18T13:52:14.045210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'173","last_scrub_stamp":"2024-08-18T13:52:14.044993+0000","last_deep_scrub":"54'173","last_deep_scrub_stamp":"2024-08-18T13:52:14.044993+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.044993+0000","objects_scrubbed":4,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:39.376618+0000","scrub_duration":0.013861017999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8216,"num_write":168,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'644","reported_seq":3302,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:40.726423+0000","last_change":"2024-08-18T13:52:40.726351+0000","last_active":"2024-08-18T13:52:40.726423+0000","last_peered":"2024-08-18T13:52:40.726423+0000","last_clean":"2024-08-18T13:52:40.726423+0000","last_became_active":"2024-08-18T13:39:39.439423+0000","last_became_peered":"2024-08-18T13:39:39.439423+0000","last_unstale":"2024-08-18T13:52:40.726423+0000","last_undegraded":"2024-08-18T13:52:40.726423+0000","last_fullsized":"2024-08-18T13:52:40.726423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'644","last_scrub_stamp":"2024-08-18T13:52:40.726235+0000","last_deep_scrub":"54'644","last_deep_scrub_stamp":"2024-08-18T13:52:40.726235+0000","last_clean_scrub_stamp":"2024-08-18T13:52:40.726235+0000","objects_scrubbed":325,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:48:53.529215+0000","scrub_duration":0.12138481800000001,"objects_trimmed":0,"snaptrim_duration":0.0059375670000000004,"stat_sum":{"num_bytes":15740586,"num_objects":325,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":572,"num_read_kb":18359,"num_write":643,"num_write_kb":23895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"54'89","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.002384+0000","last_change":"2024-08-18T13:52:14.002238+0000","last_active":"2024-08-18T13:52:14.002384+0000","last_peered":"2024-08-18T13:52:14.002384+0000","last_clean":"2024-08-18T13:52:14.002384+0000","last_became_active":"2024-08-18T13:39:35.410763+0000","last_became_peered":"2024-08-18T13:39:35.410763+0000","last_unstale":"2024-08-18T13:52:14.002384+0000","last_undegraded":"2024-08-18T13:52:14.002384+0000","last_fullsized":"2024-08-18T13:52:14.002384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'89","last_scrub_stamp":"2024-08-18T13:52:14.002012+0000","last_deep_scrub":"54'89","last_deep_scrub_stamp":"2024-08-18T13:52:14.002012+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.002012+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:43.739908+0000","scrub_duration":0.0036712979999999999,"objects_trimmed":0,"snaptrim_duration":0.0057226309999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"54'237","reported_seq":297,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.513342+0000","last_change":"2024-08-18T13:52:05.513291+0000","last_active":"2024-08-18T13:52:05.513342+0000","last_peered":"2024-08-18T13:52:05.513342+0000","last_clean":"2024-08-18T13:52:05.513342+0000","last_became_active":"2024-08-18T13:39:34.389080+0000","last_became_peered":"2024-08-18T13:39:34.389080+0000","last_unstale":"2024-08-18T13:52:05.513342+0000","last_undegraded":"2024-08-18T13:52:05.513342+0000","last_fullsized":"2024-08-18T13:52:05.513342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'237","last_scrub_stamp":"2024-08-18T13:52:05.513154+0000","last_deep_scrub":"54'237","last_deep_scrub_stamp":"2024-08-18T13:52:05.513154+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.513154+0000","objects_scrubbed":4,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T20:47:26.247014+0000","scrub_duration":0.018778702000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":20397,"num_write":232,"num_write_kb":4254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17234,"num_omap_keys":33,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"54'677","reported_seq":3193,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:54.315188+0000","last_change":"2024-08-18T13:52:54.315100+0000","last_active":"2024-08-18T13:52:54.315188+0000","last_peered":"2024-08-18T13:52:54.315188+0000","last_clean":"2024-08-18T13:52:54.315188+0000","last_became_active":"2024-08-18T13:39:39.443534+0000","last_became_peered":"2024-08-18T13:39:39.443534+0000","last_unstale":"2024-08-18T13:52:54.315188+0000","last_undegraded":"2024-08-18T13:52:54.315188+0000","last_fullsized":"2024-08-18T13:52:54.315188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'677","last_scrub_stamp":"2024-08-18T13:52:54.314944+0000","last_deep_scrub":"54'677","last_deep_scrub_stamp":"2024-08-18T13:52:54.314944+0000","last_clean_scrub_stamp":"2024-08-18T13:52:54.314944+0000","objects_scrubbed":334,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:27:33.095923+0000","scrub_duration":0.147524922,"objects_trimmed":0,"snaptrim_duration":0.0073620359999999998,"stat_sum":{"num_bytes":31240743,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":431,"num_read_kb":13577,"num_write":670,"num_write_kb":34133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"54'106","reported_seq":2181,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.455857+0000","last_change":"2024-08-18T13:52:18.455745+0000","last_active":"2024-08-18T13:52:18.455857+0000","last_peered":"2024-08-18T13:52:18.455857+0000","last_clean":"2024-08-18T13:52:18.455857+0000","last_became_active":"2024-08-18T13:39:35.406767+0000","last_became_peered":"2024-08-18T13:39:35.406767+0000","last_unstale":"2024-08-18T13:52:18.455857+0000","last_undegraded":"2024-08-18T13:52:18.455857+0000","last_fullsized":"2024-08-18T13:52:18.455857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'106","last_scrub_stamp":"2024-08-18T13:52:18.455540+0000","last_deep_scrub":"54'106","last_deep_scrub_stamp":"2024-08-18T13:52:18.455540+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.455540+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:15.305313+0000","scrub_duration":0.003717709,"objects_trimmed":0,"snaptrim_duration":0.0059613369999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"54'14","reported_seq":67,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:14.267260+0000","last_change":"2024-08-18T13:52:14.267230+0000","last_active":"2024-08-18T13:52:14.267260+0000","last_peered":"2024-08-18T13:52:14.267260+0000","last_clean":"2024-08-18T13:52:14.267260+0000","last_became_active":"2024-08-18T13:39:34.389287+0000","last_became_peered":"2024-08-18T13:39:34.389287+0000","last_unstale":"2024-08-18T13:52:14.267260+0000","last_undegraded":"2024-08-18T13:52:14.267260+0000","last_fullsized":"2024-08-18T13:52:14.267260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'14","last_scrub_stamp":"2024-08-18T13:52:14.267137+0000","last_deep_scrub":"54'14","last_deep_scrub_stamp":"2024-08-18T13:52:14.267137+0000","last_clean_scrub_stamp":"2024-08-18T13:52:14.267137+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:59.683518+0000","scrub_duration":0.010501551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":39082,"num_omap_keys":74,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"54'699","reported_seq":3505,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:42.432139+0000","last_change":"2024-08-18T13:52:42.432029+0000","last_active":"2024-08-18T13:52:42.432139+0000","last_peered":"2024-08-18T13:52:42.432139+0000","last_clean":"2024-08-18T13:52:42.432139+0000","last_became_active":"2024-08-18T13:39:39.449183+0000","last_became_peered":"2024-08-18T13:39:39.449183+0000","last_unstale":"2024-08-18T13:52:42.432139+0000","last_undegraded":"2024-08-18T13:52:42.432139+0000","last_fullsized":"2024-08-18T13:52:42.432139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'699","last_scrub_stamp":"2024-08-18T13:52:42.431869+0000","last_deep_scrub":"54'699","last_deep_scrub_stamp":"2024-08-18T13:52:42.431869+0000","last_clean_scrub_stamp":"2024-08-18T13:52:42.431869+0000","objects_scrubbed":368,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:17:40.753604+0000","scrub_duration":0.162962525,"objects_trimmed":0,"snaptrim_duration":0.0076808309999999999,"stat_sum":{"num_bytes":29636158,"num_objects":368,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":719,"num_read_kb":24704,"num_write":698,"num_write_kb":38783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"54'650","reported_seq":3316,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.124164+0000","last_change":"2024-08-18T13:52:34.124106+0000","last_active":"2024-08-18T13:52:34.124164+0000","last_peered":"2024-08-18T13:52:34.124164+0000","last_clean":"2024-08-18T13:52:34.124164+0000","last_became_active":"2024-08-18T13:39:39.453524+0000","last_became_peered":"2024-08-18T13:39:39.453524+0000","last_unstale":"2024-08-18T13:52:34.124164+0000","last_undegraded":"2024-08-18T13:52:34.124164+0000","last_fullsized":"2024-08-18T13:52:34.124164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'650","last_scrub_stamp":"2024-08-18T13:52:34.123992+0000","last_deep_scrub":"54'650","last_deep_scrub_stamp":"2024-08-18T13:52:34.123992+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.123992+0000","objects_scrubbed":334,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:16:35.992973+0000","scrub_duration":0.12763080500000001,"objects_trimmed":0,"snaptrim_duration":0.0068864499999999997,"stat_sum":{"num_bytes":24960867,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":580,"num_read_kb":20952,"num_write":650,"num_write_kb":24715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"54'91","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.205937+0000","last_change":"2024-08-18T13:52:05.205870+0000","last_active":"2024-08-18T13:52:05.205937+0000","last_peered":"2024-08-18T13:52:05.205937+0000","last_clean":"2024-08-18T13:52:05.205937+0000","last_became_active":"2024-08-18T13:39:35.401731+0000","last_became_peered":"2024-08-18T13:39:35.401731+0000","last_unstale":"2024-08-18T13:52:05.205937+0000","last_undegraded":"2024-08-18T13:52:05.205937+0000","last_fullsized":"2024-08-18T13:52:05.205937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:05.205644+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:05.205644+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.205644+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:51.244504+0000","scrub_duration":0.0033367140000000002,"objects_trimmed":0,"snaptrim_duration":0.0066095759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"54'203","reported_seq":267,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.395398+0000","last_change":"2024-08-18T13:52:10.395341+0000","last_active":"2024-08-18T13:52:10.395398+0000","last_peered":"2024-08-18T13:52:10.395398+0000","last_clean":"2024-08-18T13:52:10.395398+0000","last_became_active":"2024-08-18T13:39:34.387645+0000","last_became_peered":"2024-08-18T13:39:34.387645+0000","last_unstale":"2024-08-18T13:52:10.395398+0000","last_undegraded":"2024-08-18T13:52:10.395398+0000","last_fullsized":"2024-08-18T13:52:10.395398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'203","last_scrub_stamp":"2024-08-18T13:52:10.395155+0000","last_deep_scrub":"54'203","last_deep_scrub_stamp":"2024-08-18T13:52:10.395155+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.395155+0000","objects_scrubbed":6,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:17.573569+0000","scrub_duration":0.030766387999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":197,"num_write_kb":12441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25166,"num_omap_keys":49,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"54'658","reported_seq":3211,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.475022+0000","last_change":"2024-08-18T13:52:34.474804+0000","last_active":"2024-08-18T13:52:34.475022+0000","last_peered":"2024-08-18T13:52:34.475022+0000","last_clean":"2024-08-18T13:52:34.475022+0000","last_became_active":"2024-08-18T13:39:39.440355+0000","last_became_peered":"2024-08-18T13:39:39.440355+0000","last_unstale":"2024-08-18T13:52:34.475022+0000","last_undegraded":"2024-08-18T13:52:34.475022+0000","last_fullsized":"2024-08-18T13:52:34.475022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'658","last_scrub_stamp":"2024-08-18T13:52:34.474541+0000","last_deep_scrub":"54'658","last_deep_scrub_stamp":"2024-08-18T13:52:34.474541+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.474541+0000","objects_scrubbed":338,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:49.261380+0000","scrub_duration":0.15356631400000001,"objects_trimmed":0,"snaptrim_duration":0.0047613839999999996,"stat_sum":{"num_bytes":20867535,"num_objects":338,"num_object_clones":0,"num_object_copies":1352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":338,"num_whiteouts":0,"num_read":467,"num_read_kb":16390,"num_write":656,"num_write_kb":20726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"54'109","reported_seq":2183,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:33.293895+0000","last_change":"2024-08-18T13:52:33.293834+0000","last_active":"2024-08-18T13:52:33.293895+0000","last_peered":"2024-08-18T13:52:33.293895+0000","last_clean":"2024-08-18T13:52:33.293895+0000","last_became_active":"2024-08-18T13:39:35.411773+0000","last_became_peered":"2024-08-18T13:39:35.411773+0000","last_unstale":"2024-08-18T13:52:33.293895+0000","last_undegraded":"2024-08-18T13:52:33.293895+0000","last_fullsized":"2024-08-18T13:52:33.293895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'109","last_scrub_stamp":"2024-08-18T13:52:33.293641+0000","last_deep_scrub":"54'109","last_deep_scrub_stamp":"2024-08-18T13:52:33.293641+0000","last_clean_scrub_stamp":"2024-08-18T13:52:33.293641+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:59:10.451243+0000","scrub_duration":0.0039716550000000002,"objects_trimmed":0,"snaptrim_duration":0.0068339799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"54'105","reported_seq":163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.093354+0000","last_change":"2024-08-18T13:52:21.093276+0000","last_active":"2024-08-18T13:52:21.093354+0000","last_peered":"2024-08-18T13:52:21.093354+0000","last_clean":"2024-08-18T13:52:21.093354+0000","last_became_active":"2024-08-18T13:39:34.387141+0000","last_became_peered":"2024-08-18T13:39:34.387141+0000","last_unstale":"2024-08-18T13:52:21.093354+0000","last_undegraded":"2024-08-18T13:52:21.093354+0000","last_fullsized":"2024-08-18T13:52:21.093354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'105","last_scrub_stamp":"2024-08-18T13:52:21.093147+0000","last_deep_scrub":"54'105","last_deep_scrub_stamp":"2024-08-18T13:52:21.093147+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.093147+0000","objects_scrubbed":2,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:53:06.284734+0000","scrub_duration":0.015701764999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":103,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7766,"num_omap_keys":15,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"54'135","reported_seq":201,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.254544+0000","last_change":"2024-08-18T13:52:04.254482+0000","last_active":"2024-08-18T13:52:04.254544+0000","last_peered":"2024-08-18T13:52:04.254544+0000","last_clean":"2024-08-18T13:52:04.254544+0000","last_became_active":"2024-08-18T13:39:34.385199+0000","last_became_peered":"2024-08-18T13:39:34.385199+0000","last_unstale":"2024-08-18T13:52:04.254544+0000","last_undegraded":"2024-08-18T13:52:04.254544+0000","last_fullsized":"2024-08-18T13:52:04.254544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'135","last_scrub_stamp":"2024-08-18T13:52:04.254271+0000","last_deep_scrub":"54'135","last_deep_scrub_stamp":"2024-08-18T13:52:04.254271+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.254271+0000","objects_scrubbed":5,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:13:13.411254+0000","scrub_duration":0.03253764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":128,"num_write_kb":12417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24866,"num_omap_keys":48,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"54'91","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.023472+0000","last_change":"2024-08-18T13:52:10.023405+0000","last_active":"2024-08-18T13:52:10.023472+0000","last_peered":"2024-08-18T13:52:10.023472+0000","last_clean":"2024-08-18T13:52:10.023472+0000","last_became_active":"2024-08-18T13:39:35.407715+0000","last_became_peered":"2024-08-18T13:39:35.407715+0000","last_unstale":"2024-08-18T13:52:10.023472+0000","last_undegraded":"2024-08-18T13:52:10.023472+0000","last_fullsized":"2024-08-18T13:52:10.023472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'91","last_scrub_stamp":"2024-08-18T13:52:10.023201+0000","last_deep_scrub":"54'91","last_deep_scrub_stamp":"2024-08-18T13:52:10.023201+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.023201+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:06:06.872974+0000","scrub_duration":0.003062024,"objects_trimmed":0,"snaptrim_duration":0.0053680339999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"54'664","reported_seq":3659,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:26.626366+0000","last_change":"2024-08-18T13:52:26.626281+0000","last_active":"2024-08-18T13:52:26.626366+0000","last_peered":"2024-08-18T13:52:26.626366+0000","last_clean":"2024-08-18T13:52:26.626366+0000","last_became_active":"2024-08-18T13:39:39.436867+0000","last_became_peered":"2024-08-18T13:39:39.436867+0000","last_unstale":"2024-08-18T13:52:26.626366+0000","last_undegraded":"2024-08-18T13:52:26.626366+0000","last_fullsized":"2024-08-18T13:52:26.626366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'664","last_scrub_stamp":"2024-08-18T13:52:26.626137+0000","last_deep_scrub":"54'664","last_deep_scrub_stamp":"2024-08-18T13:52:26.626137+0000","last_clean_scrub_stamp":"2024-08-18T13:52:26.626137+0000","objects_scrubbed":365,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:57.639383+0000","scrub_duration":0.14017384899999999,"objects_trimmed":0,"snaptrim_duration":0.0035122529999999999,"stat_sum":{"num_bytes":30328916,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":908,"num_read_kb":29115,"num_write":663,"num_write_kb":34071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"54'17","reported_seq":76,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:10.959979+0000","last_change":"2024-08-18T13:52:10.959908+0000","last_active":"2024-08-18T13:52:10.959979+0000","last_peered":"2024-08-18T13:52:10.959979+0000","last_clean":"2024-08-18T13:52:10.959979+0000","last_became_active":"2024-08-18T13:39:34.397320+0000","last_became_peered":"2024-08-18T13:39:34.397320+0000","last_unstale":"2024-08-18T13:52:10.959979+0000","last_undegraded":"2024-08-18T13:52:10.959979+0000","last_fullsized":"2024-08-18T13:52:10.959979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'17","last_scrub_stamp":"2024-08-18T13:52:10.959683+0000","last_deep_scrub":"54'17","last_deep_scrub_stamp":"2024-08-18T13:52:10.959683+0000","last_clean_scrub_stamp":"2024-08-18T13:52:10.959683+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:40.697579+0000","scrub_duration":0.013613793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":19,"num_write_kb":4249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21984,"num_omap_keys":42,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:07.601341+0000","last_change":"2024-08-18T13:52:07.601203+0000","last_active":"2024-08-18T13:52:07.601341+0000","last_peered":"2024-08-18T13:52:07.601341+0000","last_clean":"2024-08-18T13:52:07.601341+0000","last_became_active":"2024-08-18T13:39:35.402866+0000","last_became_peered":"2024-08-18T13:39:35.402866+0000","last_unstale":"2024-08-18T13:52:07.601341+0000","last_undegraded":"2024-08-18T13:52:07.601341+0000","last_fullsized":"2024-08-18T13:52:07.601341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:07.600983+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:07.600983+0000","last_clean_scrub_stamp":"2024-08-18T13:52:07.600983+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:43:21.683007+0000","scrub_duration":0.0035012659999999998,"objects_trimmed":0,"snaptrim_duration":0.0072935329999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"54'656","reported_seq":3258,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:56.322683+0000","last_change":"2024-08-18T13:52:56.322632+0000","last_active":"2024-08-18T13:52:56.322683+0000","last_peered":"2024-08-18T13:52:56.322683+0000","last_clean":"2024-08-18T13:52:56.322683+0000","last_became_active":"2024-08-18T13:39:39.440677+0000","last_became_peered":"2024-08-18T13:39:39.440677+0000","last_unstale":"2024-08-18T13:52:56.322683+0000","last_undegraded":"2024-08-18T13:52:56.322683+0000","last_fullsized":"2024-08-18T13:52:56.322683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'656","last_scrub_stamp":"2024-08-18T13:52:56.322514+0000","last_deep_scrub":"54'656","last_deep_scrub_stamp":"2024-08-18T13:52:56.322514+0000","last_clean_scrub_stamp":"2024-08-18T13:52:56.322514+0000","objects_scrubbed":339,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:55:19.603412+0000","scrub_duration":0.15509093099999999,"objects_trimmed":0,"snaptrim_duration":0.0063112100000000003,"stat_sum":{"num_bytes":21201155,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":516,"num_read_kb":24278,"num_write":657,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"54'107","reported_seq":2181,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.370471+0000","last_change":"2024-08-18T13:52:21.370406+0000","last_active":"2024-08-18T13:52:21.370471+0000","last_peered":"2024-08-18T13:52:21.370471+0000","last_clean":"2024-08-18T13:52:21.370471+0000","last_became_active":"2024-08-18T13:39:35.412986+0000","last_became_peered":"2024-08-18T13:39:35.412986+0000","last_unstale":"2024-08-18T13:52:21.370471+0000","last_undegraded":"2024-08-18T13:52:21.370471+0000","last_fullsized":"2024-08-18T13:52:21.370471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'107","last_scrub_stamp":"2024-08-18T13:52:21.370198+0000","last_deep_scrub":"54'107","last_deep_scrub_stamp":"2024-08-18T13:52:21.370198+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.370198+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:44:33.767918+0000","scrub_duration":0.003909615,"objects_trimmed":0,"snaptrim_duration":0.0045422170000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"54'110","reported_seq":2184,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:21.497921+0000","last_change":"2024-08-18T13:52:21.497852+0000","last_active":"2024-08-18T13:52:21.497921+0000","last_peered":"2024-08-18T13:52:21.497921+0000","last_clean":"2024-08-18T13:52:21.497921+0000","last_became_active":"2024-08-18T13:39:35.400910+0000","last_became_peered":"2024-08-18T13:39:35.400910+0000","last_unstale":"2024-08-18T13:52:21.497921+0000","last_undegraded":"2024-08-18T13:52:21.497921+0000","last_fullsized":"2024-08-18T13:52:21.497921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'110","last_scrub_stamp":"2024-08-18T13:52:21.497653+0000","last_deep_scrub":"54'110","last_deep_scrub_stamp":"2024-08-18T13:52:21.497653+0000","last_clean_scrub_stamp":"2024-08-18T13:52:21.497653+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:53:28.676067+0000","scrub_duration":0.003899516,"objects_trimmed":0,"snaptrim_duration":0.0032950929999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"54'101","reported_seq":2185,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:17.440214+0000","last_change":"2024-08-18T13:52:17.440127+0000","last_active":"2024-08-18T13:52:17.440214+0000","last_peered":"2024-08-18T13:52:17.440214+0000","last_clean":"2024-08-18T13:52:17.440214+0000","last_became_active":"2024-08-18T13:40:16.610568+0000","last_became_peered":"2024-08-18T13:40:16.610568+0000","last_unstale":"2024-08-18T13:52:17.440214+0000","last_undegraded":"2024-08-18T13:52:17.440214+0000","last_fullsized":"2024-08-18T13:52:17.440214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'101","last_scrub_stamp":"2024-08-18T13:52:17.439868+0000","last_deep_scrub":"54'101","last_deep_scrub_stamp":"2024-08-18T13:52:17.439868+0000","last_clean_scrub_stamp":"2024-08-18T13:52:17.439868+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:06:03.535494+0000","scrub_duration":0.003706352,"objects_trimmed":0,"snaptrim_duration":0.0066503589999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"54'99","reported_seq":2174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.967255+0000","last_change":"2024-08-18T13:52:22.967191+0000","last_active":"2024-08-18T13:52:22.967255+0000","last_peered":"2024-08-18T13:52:22.967255+0000","last_clean":"2024-08-18T13:52:22.967255+0000","last_became_active":"2024-08-18T13:39:35.403683+0000","last_became_peered":"2024-08-18T13:39:35.403683+0000","last_unstale":"2024-08-18T13:52:22.967255+0000","last_undegraded":"2024-08-18T13:52:22.967255+0000","last_fullsized":"2024-08-18T13:52:22.967255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'99","last_scrub_stamp":"2024-08-18T13:52:22.966987+0000","last_deep_scrub":"54'99","last_deep_scrub_stamp":"2024-08-18T13:52:22.966987+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.966987+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:46:56.149355+0000","scrub_duration":0.0039138150000000002,"objects_trimmed":0,"snaptrim_duration":0.0056450420000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"54'80","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:05.895344+0000","last_change":"2024-08-18T13:52:05.895217+0000","last_active":"2024-08-18T13:52:05.895344+0000","last_peered":"2024-08-18T13:52:05.895344+0000","last_clean":"2024-08-18T13:52:05.895344+0000","last_became_active":"2024-08-18T13:39:35.409057+0000","last_became_peered":"2024-08-18T13:39:35.409057+0000","last_unstale":"2024-08-18T13:52:05.895344+0000","last_undegraded":"2024-08-18T13:52:05.895344+0000","last_fullsized":"2024-08-18T13:52:05.895344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'80","last_scrub_stamp":"2024-08-18T13:52:05.894985+0000","last_deep_scrub":"54'80","last_deep_scrub_stamp":"2024-08-18T13:52:05.894985+0000","last_clean_scrub_stamp":"2024-08-18T13:52:05.894985+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:51.990610+0000","scrub_duration":0.0040115990000000002,"objects_trimmed":0,"snaptrim_duration":0.0068042459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"54'74","reported_seq":2149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.251604+0000","last_change":"2024-08-18T13:52:04.251540+0000","last_active":"2024-08-18T13:52:04.251604+0000","last_peered":"2024-08-18T13:52:04.251604+0000","last_clean":"2024-08-18T13:52:04.251604+0000","last_became_active":"2024-08-18T13:39:35.402732+0000","last_became_peered":"2024-08-18T13:39:35.402732+0000","last_unstale":"2024-08-18T13:52:04.251604+0000","last_undegraded":"2024-08-18T13:52:04.251604+0000","last_fullsized":"2024-08-18T13:52:04.251604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'74","last_scrub_stamp":"2024-08-18T13:52:04.251343+0000","last_deep_scrub":"54'74","last_deep_scrub_stamp":"2024-08-18T13:52:04.251343+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.251343+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:35.264589+0000","scrub_duration":0.003478264,"objects_trimmed":0,"snaptrim_duration":0.0079157710000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"54'109","reported_seq":2183,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.271120+0000","last_change":"2024-08-18T13:52:29.271056+0000","last_active":"2024-08-18T13:52:29.271120+0000","last_peered":"2024-08-18T13:52:29.271120+0000","last_clean":"2024-08-18T13:52:29.271120+0000","last_became_active":"2024-08-18T13:39:35.411999+0000","last_became_peered":"2024-08-18T13:39:35.411999+0000","last_unstale":"2024-08-18T13:52:29.271120+0000","last_undegraded":"2024-08-18T13:52:29.271120+0000","last_fullsized":"2024-08-18T13:52:29.271120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'109","last_scrub_stamp":"2024-08-18T13:52:29.270852+0000","last_deep_scrub":"54'109","last_deep_scrub_stamp":"2024-08-18T13:52:29.270852+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.270852+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:16:03.188192+0000","scrub_duration":0.003274169,"objects_trimmed":0,"snaptrim_duration":0.0029546540000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"54'85","reported_seq":2159,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.260651+0000","last_change":"2024-08-18T13:52:29.260514+0000","last_active":"2024-08-18T13:52:29.260651+0000","last_peered":"2024-08-18T13:52:29.260651+0000","last_clean":"2024-08-18T13:52:29.260651+0000","last_became_active":"2024-08-18T13:39:35.408702+0000","last_became_peered":"2024-08-18T13:39:35.408702+0000","last_unstale":"2024-08-18T13:52:29.260651+0000","last_undegraded":"2024-08-18T13:52:29.260651+0000","last_fullsized":"2024-08-18T13:52:29.260651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:29.260293+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:29.260293+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.260293+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:16:31.129273+0000","scrub_duration":0.0036944429999999999,"objects_trimmed":0,"snaptrim_duration":0.0055234810000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"54'92","reported_seq":2161,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:28.509407+0000","last_change":"2024-08-18T13:52:28.509339+0000","last_active":"2024-08-18T13:52:28.509407+0000","last_peered":"2024-08-18T13:52:28.509407+0000","last_clean":"2024-08-18T13:52:28.509407+0000","last_became_active":"2024-08-18T13:40:16.614889+0000","last_became_peered":"2024-08-18T13:40:16.614889+0000","last_unstale":"2024-08-18T13:52:28.509407+0000","last_undegraded":"2024-08-18T13:52:28.509407+0000","last_fullsized":"2024-08-18T13:52:28.509407+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:28.509150+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:28.509150+0000","last_clean_scrub_stamp":"2024-08-18T13:52:28.509150+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:06:14.604775+0000","scrub_duration":0.0033533339999999999,"objects_trimmed":0,"snaptrim_duration":0.0033397829999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"54'92","reported_seq":2166,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:03.919727+0000","last_change":"2024-08-18T13:52:03.919668+0000","last_active":"2024-08-18T13:52:03.919727+0000","last_peered":"2024-08-18T13:52:03.919727+0000","last_clean":"2024-08-18T13:52:03.919727+0000","last_became_active":"2024-08-18T13:39:35.408648+0000","last_became_peered":"2024-08-18T13:39:35.408648+0000","last_unstale":"2024-08-18T13:52:03.919727+0000","last_undegraded":"2024-08-18T13:52:03.919727+0000","last_fullsized":"2024-08-18T13:52:03.919727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:03.919468+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:03.919468+0000","last_clean_scrub_stamp":"2024-08-18T13:52:03.919468+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:34:34.932714+0000","scrub_duration":0.0037821840000000001,"objects_trimmed":0,"snaptrim_duration":0.0070000389999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"54'95","reported_seq":2170,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.465675+0000","last_change":"2024-08-18T13:52:04.465574+0000","last_active":"2024-08-18T13:52:04.465675+0000","last_peered":"2024-08-18T13:52:04.465675+0000","last_clean":"2024-08-18T13:52:04.465675+0000","last_became_active":"2024-08-18T13:39:35.406922+0000","last_became_peered":"2024-08-18T13:39:35.406922+0000","last_unstale":"2024-08-18T13:52:04.465675+0000","last_undegraded":"2024-08-18T13:52:04.465675+0000","last_fullsized":"2024-08-18T13:52:04.465675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'95","last_scrub_stamp":"2024-08-18T13:52:04.465382+0000","last_deep_scrub":"54'95","last_deep_scrub_stamp":"2024-08-18T13:52:04.465382+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.465382+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:40:27.412242+0000","scrub_duration":0.003183946,"objects_trimmed":0,"snaptrim_duration":0.0070451519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"54'88","reported_seq":2163,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.037224+0000","last_change":"2024-08-18T13:52:30.037158+0000","last_active":"2024-08-18T13:52:30.037224+0000","last_peered":"2024-08-18T13:52:30.037224+0000","last_clean":"2024-08-18T13:52:30.037224+0000","last_became_active":"2024-08-18T13:39:35.404680+0000","last_became_peered":"2024-08-18T13:39:35.404680+0000","last_unstale":"2024-08-18T13:52:30.037224+0000","last_undegraded":"2024-08-18T13:52:30.037224+0000","last_fullsized":"2024-08-18T13:52:30.037224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:30.037003+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:30.037003+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.037003+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:04:20.479286+0000","scrub_duration":0.0027674930000000002,"objects_trimmed":0,"snaptrim_duration":0.0070254549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"54'98","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.332333+0000","last_change":"2024-08-18T13:52:22.332204+0000","last_active":"2024-08-18T13:52:22.332333+0000","last_peered":"2024-08-18T13:52:22.332333+0000","last_clean":"2024-08-18T13:52:22.332333+0000","last_became_active":"2024-08-18T13:39:35.408384+0000","last_became_peered":"2024-08-18T13:39:35.408384+0000","last_unstale":"2024-08-18T13:52:22.332333+0000","last_undegraded":"2024-08-18T13:52:22.332333+0000","last_fullsized":"2024-08-18T13:52:22.332333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'98","last_scrub_stamp":"2024-08-18T13:52:22.331993+0000","last_deep_scrub":"54'98","last_deep_scrub_stamp":"2024-08-18T13:52:22.331993+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.331993+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:49.278847+0000","scrub_duration":0.003320319,"objects_trimmed":0,"snaptrim_duration":0.003261548,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"54'85","reported_seq":2154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:44.123366+0000","last_change":"2024-08-18T13:52:44.123282+0000","last_active":"2024-08-18T13:52:44.123366+0000","last_peered":"2024-08-18T13:52:44.123366+0000","last_clean":"2024-08-18T13:52:44.123366+0000","last_became_active":"2024-08-18T13:40:16.612760+0000","last_became_peered":"2024-08-18T13:40:16.612760+0000","last_unstale":"2024-08-18T13:52:44.123366+0000","last_undegraded":"2024-08-18T13:52:44.123366+0000","last_fullsized":"2024-08-18T13:52:44.123366+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:44.123019+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:44.123019+0000","last_clean_scrub_stamp":"2024-08-18T13:52:44.123019+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T15:17:42.444754+0000","scrub_duration":0.0033194140000000001,"objects_trimmed":0,"snaptrim_duration":0.0076223990000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"54'88","reported_seq":2162,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.547065+0000","last_change":"2024-08-18T13:52:20.546999+0000","last_active":"2024-08-18T13:52:20.547065+0000","last_peered":"2024-08-18T13:52:20.547065+0000","last_clean":"2024-08-18T13:52:20.547065+0000","last_became_active":"2024-08-18T13:39:35.405130+0000","last_became_peered":"2024-08-18T13:39:35.405130+0000","last_unstale":"2024-08-18T13:52:20.547065+0000","last_undegraded":"2024-08-18T13:52:20.547065+0000","last_fullsized":"2024-08-18T13:52:20.547065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'88","last_scrub_stamp":"2024-08-18T13:52:20.546797+0000","last_deep_scrub":"54'88","last_deep_scrub_stamp":"2024-08-18T13:52:20.546797+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.546797+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:47:09.544407+0000","scrub_duration":0.0042093260000000002,"objects_trimmed":0,"snaptrim_duration":0.0027557649999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"54'94","reported_seq":2169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:16.412704+0000","last_change":"2024-08-18T13:52:16.412631+0000","last_active":"2024-08-18T13:52:16.412704+0000","last_peered":"2024-08-18T13:52:16.412704+0000","last_clean":"2024-08-18T13:52:16.412704+0000","last_became_active":"2024-08-18T13:39:35.400605+0000","last_became_peered":"2024-08-18T13:39:35.400605+0000","last_unstale":"2024-08-18T13:52:16.412704+0000","last_undegraded":"2024-08-18T13:52:16.412704+0000","last_fullsized":"2024-08-18T13:52:16.412704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'94","last_scrub_stamp":"2024-08-18T13:52:16.412429+0000","last_deep_scrub":"54'94","last_deep_scrub_stamp":"2024-08-18T13:52:16.412429+0000","last_clean_scrub_stamp":"2024-08-18T13:52:16.412429+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:05:02.451289+0000","scrub_duration":0.003524293,"objects_trimmed":0,"snaptrim_duration":0.0069473399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"54'96","reported_seq":2171,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:22.437555+0000","last_change":"2024-08-18T13:52:22.437446+0000","last_active":"2024-08-18T13:52:22.437555+0000","last_peered":"2024-08-18T13:52:22.437555+0000","last_clean":"2024-08-18T13:52:22.437555+0000","last_became_active":"2024-08-18T13:39:35.407306+0000","last_became_peered":"2024-08-18T13:39:35.407306+0000","last_unstale":"2024-08-18T13:52:22.437555+0000","last_undegraded":"2024-08-18T13:52:22.437555+0000","last_fullsized":"2024-08-18T13:52:22.437555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'96","last_scrub_stamp":"2024-08-18T13:52:22.437239+0000","last_deep_scrub":"54'96","last_deep_scrub_stamp":"2024-08-18T13:52:22.437239+0000","last_clean_scrub_stamp":"2024-08-18T13:52:22.437239+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:04:52.175134+0000","scrub_duration":0.0037466019999999999,"objects_trimmed":0,"snaptrim_duration":0.0070196240000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"54'85","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:28.954154+0000","last_change":"2024-08-18T13:52:28.954074+0000","last_active":"2024-08-18T13:52:28.954154+0000","last_peered":"2024-08-18T13:52:28.954154+0000","last_clean":"2024-08-18T13:52:28.954154+0000","last_became_active":"2024-08-18T13:39:35.397226+0000","last_became_peered":"2024-08-18T13:39:35.397226+0000","last_unstale":"2024-08-18T13:52:28.954154+0000","last_undegraded":"2024-08-18T13:52:28.954154+0000","last_fullsized":"2024-08-18T13:52:28.954154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'85","last_scrub_stamp":"2024-08-18T13:52:28.953869+0000","last_deep_scrub":"54'85","last_deep_scrub_stamp":"2024-08-18T13:52:28.953869+0000","last_clean_scrub_stamp":"2024-08-18T13:52:28.953869+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:37:06.887322+0000","scrub_duration":0.0040004230000000003,"objects_trimmed":0,"snaptrim_duration":0.004509931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"54'92","reported_seq":2167,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:18.976679+0000","last_change":"2024-08-18T13:52:18.976615+0000","last_active":"2024-08-18T13:52:18.976679+0000","last_peered":"2024-08-18T13:52:18.976679+0000","last_clean":"2024-08-18T13:52:18.976679+0000","last_became_active":"2024-08-18T13:39:35.404103+0000","last_became_peered":"2024-08-18T13:39:35.404103+0000","last_unstale":"2024-08-18T13:52:18.976679+0000","last_undegraded":"2024-08-18T13:52:18.976679+0000","last_fullsized":"2024-08-18T13:52:18.976679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'92","last_scrub_stamp":"2024-08-18T13:52:18.976444+0000","last_deep_scrub":"54'92","last_deep_scrub_stamp":"2024-08-18T13:52:18.976444+0000","last_clean_scrub_stamp":"2024-08-18T13:52:18.976444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:27:33.305679+0000","scrub_duration":0.0033123919999999999,"objects_trimmed":0,"snaptrim_duration":0.0062343609999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"54'81","reported_seq":2155,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:37.251183+0000","last_change":"2024-08-18T13:52:37.251112+0000","last_active":"2024-08-18T13:52:37.251183+0000","last_peered":"2024-08-18T13:52:37.251183+0000","last_clean":"2024-08-18T13:52:37.251183+0000","last_became_active":"2024-08-18T13:39:35.404535+0000","last_became_peered":"2024-08-18T13:39:35.404535+0000","last_unstale":"2024-08-18T13:52:37.251183+0000","last_undegraded":"2024-08-18T13:52:37.251183+0000","last_fullsized":"2024-08-18T13:52:37.251183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'81","last_scrub_stamp":"2024-08-18T13:52:37.250900+0000","last_deep_scrub":"54'81","last_deep_scrub_stamp":"2024-08-18T13:52:37.250900+0000","last_clean_scrub_stamp":"2024-08-18T13:52:37.250900+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T16:56:39.825671+0000","scrub_duration":0.0040894440000000002,"objects_trimmed":0,"snaptrim_duration":0.0072121939999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"54'77","reported_seq":2152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:17.643221+0000","last_change":"2024-08-18T13:52:17.643163+0000","last_active":"2024-08-18T13:52:17.643221+0000","last_peered":"2024-08-18T13:52:17.643221+0000","last_clean":"2024-08-18T13:52:17.643221+0000","last_became_active":"2024-08-18T13:39:35.405464+0000","last_became_peered":"2024-08-18T13:39:35.405464+0000","last_unstale":"2024-08-18T13:52:17.643221+0000","last_undegraded":"2024-08-18T13:52:17.643221+0000","last_fullsized":"2024-08-18T13:52:17.643221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'77","last_scrub_stamp":"2024-08-18T13:52:17.642974+0000","last_deep_scrub":"54'77","last_deep_scrub_stamp":"2024-08-18T13:52:17.642974+0000","last_clean_scrub_stamp":"2024-08-18T13:52:17.642974+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:17.078190+0000","scrub_duration":0.0036672470000000002,"objects_trimmed":0,"snaptrim_duration":0.0057680220000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"54'90","reported_seq":2164,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:25.323806+0000","last_change":"2024-08-18T13:52:25.323732+0000","last_active":"2024-08-18T13:52:25.323806+0000","last_peered":"2024-08-18T13:52:25.323806+0000","last_clean":"2024-08-18T13:52:25.323806+0000","last_became_active":"2024-08-18T13:39:35.410860+0000","last_became_peered":"2024-08-18T13:39:35.410860+0000","last_unstale":"2024-08-18T13:52:25.323806+0000","last_undegraded":"2024-08-18T13:52:25.323806+0000","last_fullsized":"2024-08-18T13:52:25.323806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:25.323493+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:25.323493+0000","last_clean_scrub_stamp":"2024-08-18T13:52:25.323493+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:19:16.878783+0000","scrub_duration":0.0036337510000000002,"objects_trimmed":0,"snaptrim_duration":0.0049025989999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"54'98","reported_seq":2172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:30.239723+0000","last_change":"2024-08-18T13:52:30.239659+0000","last_active":"2024-08-18T13:52:30.239723+0000","last_peered":"2024-08-18T13:52:30.239723+0000","last_clean":"2024-08-18T13:52:30.239723+0000","last_became_active":"2024-08-18T13:39:35.417755+0000","last_became_peered":"2024-08-18T13:39:35.417755+0000","last_unstale":"2024-08-18T13:52:30.239723+0000","last_undegraded":"2024-08-18T13:52:30.239723+0000","last_fullsized":"2024-08-18T13:52:30.239723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'98","last_scrub_stamp":"2024-08-18T13:52:30.239450+0000","last_deep_scrub":"54'98","last_deep_scrub_stamp":"2024-08-18T13:52:30.239450+0000","last_clean_scrub_stamp":"2024-08-18T13:52:30.239450+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T22:32:05.377873+0000","scrub_duration":0.0037021020000000001,"objects_trimmed":0,"snaptrim_duration":0.0029569129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"54'83","reported_seq":2157,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:34.837796+0000","last_change":"2024-08-18T13:52:34.837716+0000","last_active":"2024-08-18T13:52:34.837796+0000","last_peered":"2024-08-18T13:52:34.837796+0000","last_clean":"2024-08-18T13:52:34.837796+0000","last_became_active":"2024-08-18T13:39:35.421346+0000","last_became_peered":"2024-08-18T13:39:35.421346+0000","last_unstale":"2024-08-18T13:52:34.837796+0000","last_undegraded":"2024-08-18T13:52:34.837796+0000","last_fullsized":"2024-08-18T13:52:34.837796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'83","last_scrub_stamp":"2024-08-18T13:52:34.837509+0000","last_deep_scrub":"54'83","last_deep_scrub_stamp":"2024-08-18T13:52:34.837509+0000","last_clean_scrub_stamp":"2024-08-18T13:52:34.837509+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:56:49.624348+0000","scrub_duration":0.0037147289999999999,"objects_trimmed":0,"snaptrim_duration":0.0062586680000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"54'74","reported_seq":2148,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:04.923219+0000","last_change":"2024-08-18T13:52:04.923145+0000","last_active":"2024-08-18T13:52:04.923219+0000","last_peered":"2024-08-18T13:52:04.923219+0000","last_clean":"2024-08-18T13:52:04.923219+0000","last_became_active":"2024-08-18T13:39:35.411157+0000","last_became_peered":"2024-08-18T13:39:35.411157+0000","last_unstale":"2024-08-18T13:52:04.923219+0000","last_undegraded":"2024-08-18T13:52:04.923219+0000","last_fullsized":"2024-08-18T13:52:04.923219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'74","last_scrub_stamp":"2024-08-18T13:52:04.922933+0000","last_deep_scrub":"54'74","last_deep_scrub_stamp":"2024-08-18T13:52:04.922933+0000","last_clean_scrub_stamp":"2024-08-18T13:52:04.922933+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T17:04:50.961793+0000","scrub_duration":0.0036959599999999999,"objects_trimmed":0,"snaptrim_duration":0.0064056340000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"54'105","reported_seq":2179,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:31.843758+0000","last_change":"2024-08-18T13:52:31.843695+0000","last_active":"2024-08-18T13:52:31.843758+0000","last_peered":"2024-08-18T13:52:31.843758+0000","last_clean":"2024-08-18T13:52:31.843758+0000","last_became_active":"2024-08-18T13:39:35.415251+0000","last_became_peered":"2024-08-18T13:39:35.415251+0000","last_unstale":"2024-08-18T13:52:31.843758+0000","last_undegraded":"2024-08-18T13:52:31.843758+0000","last_fullsized":"2024-08-18T13:52:31.843758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'105","last_scrub_stamp":"2024-08-18T13:52:31.843506+0000","last_deep_scrub":"54'105","last_deep_scrub_stamp":"2024-08-18T13:52:31.843506+0000","last_clean_scrub_stamp":"2024-08-18T13:52:31.843506+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:54:05.314296+0000","scrub_duration":0.003779241,"objects_trimmed":0,"snaptrim_duration":0.0050911749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"54'93","reported_seq":2168,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:25.511140+0000","last_change":"2024-08-18T13:52:25.511071+0000","last_active":"2024-08-18T13:52:25.511140+0000","last_peered":"2024-08-18T13:52:25.511140+0000","last_clean":"2024-08-18T13:52:25.511140+0000","last_became_active":"2024-08-18T13:39:35.407548+0000","last_became_peered":"2024-08-18T13:39:35.407548+0000","last_unstale":"2024-08-18T13:52:25.511140+0000","last_undegraded":"2024-08-18T13:52:25.511140+0000","last_fullsized":"2024-08-18T13:52:25.511140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'93","last_scrub_stamp":"2024-08-18T13:52:25.510879+0000","last_deep_scrub":"54'93","last_deep_scrub_stamp":"2024-08-18T13:52:25.510879+0000","last_clean_scrub_stamp":"2024-08-18T13:52:25.510879+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T14:32:50.842504+0000","scrub_duration":0.0038326670000000001,"objects_trimmed":0,"snaptrim_duration":0.0068654359999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"54'102","reported_seq":2176,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.859948+0000","last_change":"2024-08-18T13:52:29.859863+0000","last_active":"2024-08-18T13:52:29.859948+0000","last_peered":"2024-08-18T13:52:29.859948+0000","last_clean":"2024-08-18T13:52:29.859948+0000","last_became_active":"2024-08-18T13:39:35.416700+0000","last_became_peered":"2024-08-18T13:39:35.416700+0000","last_unstale":"2024-08-18T13:52:29.859948+0000","last_undegraded":"2024-08-18T13:52:29.859948+0000","last_fullsized":"2024-08-18T13:52:29.859948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'102","last_scrub_stamp":"2024-08-18T13:52:29.859674+0000","last_deep_scrub":"54'102","last_deep_scrub_stamp":"2024-08-18T13:52:29.859674+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.859674+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T13:58:17.203374+0000","scrub_duration":0.0033213359999999998,"objects_trimmed":0,"snaptrim_duration":0.0055002549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"54'76","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:29.564884+0000","last_change":"2024-08-18T13:52:29.564814+0000","last_active":"2024-08-18T13:52:29.564884+0000","last_peered":"2024-08-18T13:52:29.564884+0000","last_clean":"2024-08-18T13:52:29.564884+0000","last_became_active":"2024-08-18T13:39:35.409703+0000","last_became_peered":"2024-08-18T13:39:35.409703+0000","last_unstale":"2024-08-18T13:52:29.564884+0000","last_undegraded":"2024-08-18T13:52:29.564884+0000","last_fullsized":"2024-08-18T13:52:29.564884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'76","last_scrub_stamp":"2024-08-18T13:52:29.564588+0000","last_deep_scrub":"54'76","last_deep_scrub_stamp":"2024-08-18T13:52:29.564588+0000","last_clean_scrub_stamp":"2024-08-18T13:52:29.564588+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:32.503951+0000","scrub_duration":0.0037994510000000001,"objects_trimmed":0,"snaptrim_duration":0.0069338170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"54'90","reported_seq":2165,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:20.940841+0000","last_change":"2024-08-18T13:52:20.940751+0000","last_active":"2024-08-18T13:52:20.940841+0000","last_peered":"2024-08-18T13:52:20.940841+0000","last_clean":"2024-08-18T13:52:20.940841+0000","last_became_active":"2024-08-18T13:39:35.404352+0000","last_became_peered":"2024-08-18T13:39:35.404352+0000","last_unstale":"2024-08-18T13:52:20.940841+0000","last_undegraded":"2024-08-18T13:52:20.940841+0000","last_fullsized":"2024-08-18T13:52:20.940841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'90","last_scrub_stamp":"2024-08-18T13:52:20.940535+0000","last_deep_scrub":"54'90","last_deep_scrub_stamp":"2024-08-18T13:52:20.940535+0000","last_clean_scrub_stamp":"2024-08-18T13:52:20.940535+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T18:04:23.879899+0000","scrub_duration":0.003381294,"objects_trimmed":0,"snaptrim_duration":0.0044622389999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"54'76","reported_seq":2151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:27.995177+0000","last_change":"2024-08-18T13:52:27.995104+0000","last_active":"2024-08-18T13:52:27.995177+0000","last_peered":"2024-08-18T13:52:27.995177+0000","last_clean":"2024-08-18T13:52:27.995177+0000","last_became_active":"2024-08-18T13:39:35.405343+0000","last_became_peered":"2024-08-18T13:39:35.405343+0000","last_unstale":"2024-08-18T13:52:27.995177+0000","last_undegraded":"2024-08-18T13:52:27.995177+0000","last_fullsized":"2024-08-18T13:52:27.995177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'76","last_scrub_stamp":"2024-08-18T13:52:27.994891+0000","last_deep_scrub":"54'76","last_deep_scrub_stamp":"2024-08-18T13:52:27.994891+0000","last_clean_scrub_stamp":"2024-08-18T13:52:27.994891+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T21:32:36.746461+0000","scrub_duration":0.0032603279999999998,"objects_trimmed":0,"snaptrim_duration":0.0035965799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"54'86","reported_seq":2160,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-08-18T13:52:42.881299+0000","last_change":"2024-08-18T13:52:42.881234+0000","last_active":"2024-08-18T13:52:42.881299+0000","last_peered":"2024-08-18T13:52:42.881299+0000","last_clean":"2024-08-18T13:52:42.881299+0000","last_became_active":"2024-08-18T13:39:35.416902+0000","last_became_peered":"2024-08-18T13:39:35.416902+0000","last_unstale":"2024-08-18T13:52:42.881299+0000","last_undegraded":"2024-08-18T13:52:42.881299+0000","last_fullsized":"2024-08-18T13:52:42.881299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'86","last_scrub_stamp":"2024-08-18T13:52:42.881004+0000","last_deep_scrub":"54'86","last_deep_scrub_stamp":"2024-08-18T13:52:42.881004+0000","last_clean_scrub_stamp":"2024-08-18T13:52:42.881004+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T19:23:19.221237+0000","scrub_duration":0.0032102340000000002,"objects_trimmed":0,"snaptrim_duration":0.0067690759999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":938443088,"num_objects":10906,"num_object_clones":0,"num_object_copies":43624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10906,"num_whiteouts":0,"num_read":21895,"num_read_kb":705026,"num_write":21484,"num_write_kb":1061752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2001649664,"data_stored":2001649664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21526,"ondisk_log_size":21526,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5769,"ondisk_log_size":5769,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":184848248,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":586,"num_read_kb":685208,"num_write":6435,"num_write_kb":186300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":79,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576095,"num_omap_keys":1100,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":369762304,"data_stored":369696496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6483,"ondisk_log_size":6483,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509608,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600108,"kb_used_data":294436,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93771732,"statfs":{"total":96636764160,"available":96022253568,"internally_reserved":0,"allocated":301502464,"data_stored":301200284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.0649999999999999,"15min":0.88100000000000001},"min":{"1min":0.54100000000000004,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":1.254,"5min":25.744,"15min":25.744},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86299999999999999,"15min":0.82499999999999996},"min":{"1min":0.57299999999999995,"5min":0.40300000000000002,"15min":0.33100000000000002},"max":{"1min":1.234,"5min":8.1679999999999993,"15min":8.1679999999999993},"last":1.0549999999999999}]},{"osd":1,"last update":"Sun Aug 18 13:52:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90900000000000003,"15min":0.98499999999999999},"min":{"1min":0.57099999999999995,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.171,"5min":10.151999999999999,"15min":39.203000000000003},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.1319999999999999,"15min":0.95499999999999996},"min":{"1min":0.56000000000000005,"5min":0.39300000000000002,"15min":0.34899999999999998},"max":{"1min":1.2090000000000001,"5min":25.852,"15min":25.852},"last":0.81100000000000005}]},{"osd":2,"last update":"Sun Aug 18 13:52:56 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.1499999999999999,"15min":0.95399999999999996},"min":{"1min":0.623,"5min":0.435,"15min":0.39400000000000002},"max":{"1min":1.4159999999999999,"5min":25.818999999999999,"15min":25.818999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86699999999999999,"15min":0.85399999999999998},"min":{"1min":0.629,"5min":0.42299999999999999,"15min":0.35799999999999998},"max":{"1min":1.173,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.752}]},{"osd":3,"last update":"Sun Aug 18 13:52:56 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.0660000000000001,"15min":1.1319999999999999},"min":{"1min":0.59799999999999998,"5min":0.42699999999999999,"15min":0.377},"max":{"1min":1.147,"5min":17.300999999999998,"15min":42.905000000000001},"last":1.147},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.2569999999999999,"15min":1.1180000000000001},"min":{"1min":0.873,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.339,"5min":25.786999999999999,"15min":25.786999999999999},"last":0.97499999999999998}]},{"osd":4,"last update":"Sun Aug 18 13:52:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.94799999999999995,"15min":0.93300000000000005},"min":{"1min":0.48499999999999999,"5min":0.376,"15min":0.29699999999999999},"max":{"1min":1.083,"5min":8.3870000000000005,"15min":8.3870000000000005},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.008,"5min":1.1599999999999999,"15min":1.2290000000000001},"min":{"1min":0.56100000000000005,"5min":0.371,"15min":0.35799999999999998},"max":{"1min":1.2849999999999999,"5min":17.056999999999999,"15min":52.673000000000002},"last":1.2849999999999999}]},{"osd":5,"last update":"Sun Aug 18 13:52:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.2749999999999999,"15min":1.1160000000000001},"min":{"1min":0.55200000000000005,"5min":0.44,"15min":0.33700000000000002},"max":{"1min":1.256,"5min":25.881,"15min":25.881},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.095,"15min":1.0640000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.37},"max":{"1min":1.347,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.218}]},{"osd":6,"last update":"Sun Aug 18 13:52:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.157,"15min":1.119},"min":{"1min":0.83499999999999996,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":1.325,"5min":8.0510000000000002,"15min":8.0510000000000002},"last":1.325},{"interface":"front","average":{"1min":1.075,"5min":1.1919999999999999,"15min":1.2549999999999999},"min":{"1min":0.74199999999999999,"5min":0.49399999999999999,"15min":0.435},"max":{"1min":1.4059999999999999,"5min":10.523,"15min":39.161000000000001},"last":1.379}]}]},{"osd":6,"up_from":15,"seq":64424509609,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610072,"kb_used_data":312720,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93761768,"statfs":{"total":96636764160,"available":96012050432,"internally_reserved":0,"allocated":320225280,"data_stored":319926507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.77000000000000002,"15min":0.86599999999999999},"min":{"1min":0.44400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":4.532},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79000000000000004,"15min":0.83099999999999996},"min":{"1min":0.55300000000000005,"5min":0.34100000000000003,"15min":0.248},"max":{"1min":1.2030000000000001,"5min":1.3200000000000001,"15min":1.9650000000000001},"last":0.77700000000000002}]},{"osd":1,"last update":"Sun Aug 18 13:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.84699999999999998,"15min":0.91700000000000004},"min":{"1min":0.51900000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.337,"5min":1.337,"15min":4.468},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.84499999999999997,"15min":0.91100000000000003},"min":{"1min":0.64400000000000002,"5min":0.38300000000000001,"15min":0.30499999999999999},"max":{"1min":1.325,"5min":1.325,"15min":2.3620000000000001},"last":1.052}]},{"osd":2,"last update":"Sun Aug 18 13:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.78900000000000003,"15min":0.83599999999999997},"min":{"1min":0.66200000000000003,"5min":0.40200000000000002,"15min":0.28000000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.413},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.89400000000000002,"15min":0.96899999999999997},"min":{"1min":0.56000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.218,"5min":1.468,"15min":4.5},"last":1.1080000000000001}]},{"osd":3,"last update":"Sun Aug 18 13:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.73599999999999999,"5min":0.41999999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.276,"15min":4.3250000000000002},"last":1.222},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.88400000000000001,"15min":0.91400000000000003},"min":{"1min":0.623,"5min":0.47799999999999998,"15min":0.32900000000000001},"max":{"1min":1.1579999999999999,"5min":1.306,"15min":1.8069999999999999},"last":1.2549999999999999}]},{"osd":4,"last update":"Sun Aug 18 13:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.92200000000000004,"15min":1},"min":{"1min":0.59399999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3240000000000001,"5min":1.552,"15min":5.2549999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.98699999999999999,"15min":1.0189999999999999},"min":{"1min":0.77400000000000002,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":1.4219999999999999,"5min":1.7230000000000001,"15min":4.6600000000000001},"last":1.3}]},{"osd":5,"last update":"Sun Aug 18 13:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":0.93100000000000005,"15min":1.004},"min":{"1min":0.84599999999999997,"5min":0.45400000000000001,"15min":0.373},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":4.4009999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.05,"15min":1.1399999999999999},"min":{"1min":0.68000000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4079999999999999,"5min":1.6879999999999999,"15min":6.1260000000000003},"last":1.1890000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0740000000000001,"15min":1.103},"min":{"1min":0.89500000000000002,"5min":0.55500000000000005,"15min":0.40500000000000003},"max":{"1min":1.3340000000000001,"5min":1.4299999999999999,"15min":1.9299999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.069,"15min":1.1000000000000001},"min":{"1min":0.70599999999999996,"5min":0.46300000000000002,"15min":0.40200000000000002},"max":{"1min":1.365,"5min":1.8,"15min":2.1150000000000002},"last":1.385}]}]},{"osd":4,"up_from":15,"seq":64424509607,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570388,"kb_used_data":273036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93801452,"statfs":{"total":96636764160,"available":96052686848,"internally_reserved":0,"allocated":279588864,"data_stored":279281600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.745,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.34000000000000002,"15min":0.24299999999999999},"max":{"1min":1.0509999999999999,"5min":1.204,"15min":1.22},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.83299999999999996,"15min":0.84399999999999997},"min":{"1min":0.47899999999999998,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":1.2749999999999999,"5min":1.339,"15min":1.73},"last":0.97199999999999998}]},{"osd":1,"last update":"Sun Aug 18 13:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83199999999999996,"15min":0.85899999999999999},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":1.2150000000000001,"5min":1.415,"15min":3.6299999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85299999999999998,"15min":0.85799999999999998},"min":{"1min":0.61899999999999999,"5min":0.36699999999999999,"15min":0.27600000000000002},"max":{"1min":1.194,"5min":1.25,"15min":1.5169999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Aug 18 13:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.83799999999999997,"15min":0.85599999999999998},"min":{"1min":0.66900000000000004,"5min":0.45600000000000002,"15min":0.40000000000000002},"max":{"1min":1.2669999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.91800000000000004,"15min":0.92500000000000004},"min":{"1min":0.77000000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.2470000000000001,"5min":1.4039999999999999,"15min":3.5920000000000001},"last":0.878}]},{"osd":3,"last update":"Sun Aug 18 13:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.89900000000000002,"15min":0.91400000000000003},"min":{"1min":0.71099999999999997,"5min":0.498,"15min":0.309},"max":{"1min":1.222,"5min":1.3680000000000001,"15min":1.45},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.94199999999999995,"15min":0.97299999999999998},"min":{"1min":0.70599999999999996,"5min":0.51900000000000002,"15min":0.41799999999999998},"max":{"1min":1.1619999999999999,"5min":1.641,"15min":1.641},"last":1.1279999999999999}]},{"osd":5,"last update":"Sun Aug 18 13:52:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92400000000000004,"15min":0.96399999999999997},"min":{"1min":0.622,"5min":0.42499999999999999,"15min":0.313},"max":{"1min":1.2390000000000001,"5min":1.575,"15min":3.7789999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.98699999999999999,"15min":1.0109999999999999},"min":{"1min":0.754,"5min":0.436,"15min":0.372},"max":{"1min":1.3080000000000001,"5min":1.754,"15min":1.754},"last":1.1950000000000001}]},{"osd":6,"last update":"Sun Aug 18 13:52:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.038,"15min":1.0740000000000001},"min":{"1min":0.68300000000000005,"5min":0.51500000000000001,"15min":0.504},"max":{"1min":1.246,"5min":1.667,"15min":7.8490000000000002},"last":1.101},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.998,"15min":1.0569999999999999},"min":{"1min":0.752,"5min":0.437,"15min":0.437},"max":{"1min":1.282,"5min":1.47,"15min":6.3380000000000001},"last":1.046}]},{"osd":7,"last update":"Sun Aug 18 13:52:47 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.0589999999999999,"15min":1.089},"min":{"1min":0.90600000000000003,"5min":0.46500000000000002,"15min":0.40000000000000002},"max":{"1min":1.3160000000000001,"5min":1.7470000000000001,"15min":1.8400000000000001},"last":1.075},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1399999999999999,"15min":1.177},"min":{"1min":1.0169999999999999,"5min":0.55900000000000005,"15min":0.42399999999999999},"max":{"1min":1.4299999999999999,"5min":1.722,"15min":3.7410000000000001},"last":1.1719999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542312,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683436,"kb_used_data":373604,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93688404,"statfs":{"total":96636764160,"available":95936925696,"internally_reserved":0,"allocated":382570496,"data_stored":382269906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.66600000000000004,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.192,"15min":0.192},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.70699999999999996,"15min":0.70399999999999996},"min":{"1min":0.56599999999999995,"5min":0.29299999999999998,"15min":0.28000000000000003},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.79700000000000004}]},{"osd":1,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.442,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":3.4820000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.76700000000000002,"15min":0.76100000000000001},"min":{"1min":0.58399999999999996,"5min":0.38,"15min":0.30499999999999999},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":0.76700000000000002}]},{"osd":2,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.61199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.046,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.66500000000000004,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.83099999999999996}]},{"osd":4,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.93000000000000005,"15min":0.96799999999999997},"min":{"1min":0.63500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":3.2490000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.028,"15min":1.0529999999999999},"min":{"1min":0.85699999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.1850000000000001,"5min":3.1850000000000001,"15min":3.8799999999999999},"last":0.89100000000000001}]},{"osd":5,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97199999999999998,"15min":0.99299999999999999},"min":{"1min":0.57299999999999995,"5min":0.51500000000000001,"15min":0.38700000000000001},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":4.1059999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.022,"15min":1.0549999999999999},"min":{"1min":0.70899999999999996,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.2799999999999998},"last":1.1319999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.028,"15min":1.036},"min":{"1min":0.69799999999999995,"5min":0.504,"15min":0.41999999999999998},"max":{"1min":3.2599999999999998,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.073,"15min":1.0980000000000001},"min":{"1min":0.77700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":1.1779999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.123,"15min":1.1299999999999999},"min":{"1min":0.83799999999999997,"5min":0.64900000000000002,"15min":0.51900000000000002},"max":{"1min":3.306,"5min":3.306,"15min":3.306},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.105,"15min":1.1040000000000001},"min":{"1min":0.64300000000000002,"5min":0.63200000000000001,"15min":0.51100000000000001},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.153}]}]},{"osd":2,"up_from":14,"seq":60129542311,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530780,"kb_used_data":229268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93841060,"statfs":{"total":96636764160,"available":96093245440,"internally_reserved":0,"allocated":234770432,"data_stored":234477374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59799999999999998,"15min":0.61499999999999999},"min":{"1min":0.38700000000000001,"5min":0.24099999999999999,"15min":0.186},"max":{"1min":0.94499999999999995,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.34799999999999998,"5min":0.29499999999999998,"15min":0.26400000000000001},"max":{"1min":0.83599999999999997,"5min":3.27,"15min":3.27},"last":0.81799999999999995}]},{"osd":1,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.64300000000000002,"15min":0.68899999999999995},"min":{"1min":0.436,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93799999999999994,"5min":1.038,"15min":1.696},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70599999999999996,"15min":0.73899999999999999},"min":{"1min":0.436,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.89400000000000002,"5min":3.2290000000000001,"15min":3.2290000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66500000000000004,"15min":0.77800000000000002},"min":{"1min":0.40999999999999998,"5min":0.28899999999999998,"15min":0.253},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":12.468999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81399999999999995,"15min":0.84099999999999997},"min":{"1min":0.54800000000000004,"5min":0.35899999999999999,"15min":0.27500000000000002},"max":{"1min":1.105,"5min":1.3540000000000001,"15min":1.744},"last":0.84799999999999998}]},{"osd":4,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.91900000000000004,"15min":0.95499999999999996},"min":{"1min":0.61199999999999999,"5min":0.49299999999999999,"15min":0.40100000000000002},"max":{"1min":1.208,"5min":1.274,"15min":1.8480000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.94799999999999995},"min":{"1min":0.58399999999999996,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.117,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.92000000000000004}]},{"osd":5,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.93600000000000005,"15min":0.95599999999999996},"min":{"1min":0.63200000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.113,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91600000000000004,"15min":0.94999999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.39000000000000001},"max":{"1min":1.1910000000000001,"5min":1.365,"15min":1.627},"last":1.0229999999999999}]},{"osd":6,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.93600000000000005,"15min":0.97899999999999998},"min":{"1min":0.55500000000000005,"5min":0.499,"15min":0.42199999999999999},"max":{"1min":1.2729999999999999,"5min":1.413,"15min":1.649},"last":1.226},{"interface":"front","average":{"1min":1.093,"5min":1.0189999999999999,"15min":1.0620000000000001},"min":{"1min":0.77500000000000002,"5min":0.51900000000000002,"15min":0.49099999999999999},"max":{"1min":1.3140000000000001,"5min":1.3220000000000001,"15min":1.905},"last":1.1779999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.05,"15min":1.0860000000000001},"min":{"1min":0.78900000000000003,"5min":0.60399999999999998,"15min":0.436},"max":{"1min":1.335,"5min":1.3999999999999999,"15min":1.9490000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.98799999999999999,"15min":1.014},"min":{"1min":0.65300000000000002,"5min":0.53200000000000003,"15min":0.46400000000000002},"max":{"1min":1.3240000000000001,"5min":1.4379999999999999,"15min":1.7669999999999999},"last":1.1240000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542311,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677976,"kb_used_data":376464,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93693864,"statfs":{"total":96636764160,"available":95942516736,"internally_reserved":0,"allocated":385499136,"data_stored":385199863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.64400000000000002},"min":{"1min":0.40799999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.747,"5min":1.3009999999999999,"15min":3.1160000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.64500000000000002,"15min":0.71399999999999997},"min":{"1min":0.433,"5min":0.251,"15min":0.251},"max":{"1min":1.0940000000000001,"5min":1.107,"15min":3.718},"last":0.61699999999999999}]},{"osd":2,"last update":"Sun Aug 18 13:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.29799999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1990000000000001,"5min":4.968,"15min":4.968},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74399999999999999,"15min":0.79600000000000004},"min":{"1min":0.45600000000000002,"5min":0.33400000000000002,"15min":0.20899999999999999},"max":{"1min":1.008,"5min":1.327,"15min":9.3309999999999995},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Aug 18 13:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.80600000000000005,"15min":0.82499999999999996},"min":{"1min":0.46700000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.79800000000000004,"15min":0.84799999999999998},"min":{"1min":0.504,"5min":0.40699999999999997,"15min":0.35199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":6.8559999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Sun Aug 18 13:52:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.94499999999999995,"15min":0.97599999999999998},"min":{"1min":0.57099999999999995,"5min":0.44500000000000001,"15min":0.28000000000000003},"max":{"1min":1.4630000000000001,"5min":5.1020000000000003,"15min":5.6529999999999996},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99199999999999999,"15min":1.0269999999999999},"min":{"1min":0.57199999999999995,"5min":0.39700000000000002,"15min":0.28599999999999998},"max":{"1min":1.22,"5min":1.4410000000000001,"15min":1.5740000000000001},"last":1.139}]},{"osd":5,"last update":"Sun Aug 18 13:52:42 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.99099999999999999,"15min":1.0469999999999999},"min":{"1min":0.755,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.343,"5min":1.466,"15min":3.7570000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0349999999999999,"15min":1.1020000000000001},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.45700000000000002},"max":{"1min":1.385,"5min":5.1799999999999997,"15min":19.027999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Sun Aug 18 13:52:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.071},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.40699999999999997},"max":{"1min":1.446,"5min":5.2480000000000002,"15min":5.2480000000000002},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0680000000000001,"15min":1.121},"min":{"1min":0.72699999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"max":{"1min":1.3,"5min":1.599,"15min":1.861},"last":1.0409999999999999}]},{"osd":7,"last update":"Sun Aug 18 13:52:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0580000000000001,"15min":1.119},"min":{"1min":0.77700000000000002,"5min":0.60299999999999998,"15min":0.58799999999999997},"max":{"1min":1.712,"5min":1.712,"15min":3.8100000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.40000000000000002},"max":{"1min":1.827,"5min":5.3099999999999996,"15min":5.3099999999999996},"last":1.087}]}]},{"osd":0,"up_from":14,"seq":60129542311,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549780,"kb_used_data":248268,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93822060,"statfs":{"total":96636764160,"available":96073789440,"internally_reserved":0,"allocated":254226432,"data_stored":253919977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Aug 18 13:52:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.65100000000000002},"min":{"1min":0.27700000000000002,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":5.3159999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70199999999999996,"15min":0.72199999999999998},"min":{"1min":0.38700000000000001,"5min":0.254,"15min":0.19500000000000001},"max":{"1min":1.1319999999999999,"5min":1.2370000000000001,"15min":3.4809999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Sun Aug 18 13:52:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71999999999999997,"15min":0.72999999999999998},"min":{"1min":0.435,"5min":0.32600000000000001,"15min":0.26300000000000001},"max":{"1min":0.996,"5min":1.222,"15min":3.3029999999999999},"last":1.075},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64800000000000002,"15min":0.67700000000000005},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.97299999999999998,"5min":1.135,"15min":1.28},"last":0.65400000000000003}]},{"osd":3,"last update":"Sun Aug 18 13:52:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.753,"15min":0.81799999999999995},"min":{"1min":0.52400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.026,"5min":1.288,"15min":5.399},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.872,"15min":0.93000000000000005},"min":{"1min":0.59099999999999997,"5min":0.41599999999999998,"15min":0.33900000000000002},"max":{"1min":1.1419999999999999,"5min":1.365,"15min":5.0359999999999996},"last":1.173}]},{"osd":4,"last update":"Sun Aug 18 13:52:54 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.88300000000000001,"15min":0.91900000000000004},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.37},"max":{"1min":1.3089999999999999,"5min":1.3899999999999999,"15min":1.6020000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.97899999999999998,"15min":1.044},"min":{"1min":0.65300000000000002,"5min":0.50800000000000001,"15min":0.35099999999999998},"max":{"1min":1.264,"5min":1.6559999999999999,"15min":8.7799999999999994},"last":1.0409999999999999}]},{"osd":5,"last update":"Sun Aug 18 13:52:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.94499999999999995,"15min":0.97499999999999998},"min":{"1min":0.68300000000000005,"5min":0.49299999999999999,"15min":0.35499999999999998},"max":{"1min":1.377,"5min":1.5940000000000001,"15min":1.7150000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.036,"15min":1.0760000000000001},"min":{"1min":0.56299999999999994,"5min":0.43099999999999999,"15min":0.40300000000000002},"max":{"1min":1.347,"5min":1.51,"15min":1.8879999999999999},"last":1.109}]},{"osd":6,"last update":"Sun Aug 18 13:52:54 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.0640000000000001,"15min":1.103},"min":{"1min":0.81499999999999995,"5min":0.46500000000000002,"15min":0.374},"max":{"1min":1.5409999999999999,"5min":1.599,"15min":3.7200000000000002},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.036,"15min":1.085},"min":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.52600000000000002},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":3.2789999999999999},"last":1.2430000000000001}]},{"osd":7,"last update":"Sun Aug 18 13:52:54 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1240000000000001,"15min":1.274},"min":{"1min":0.77200000000000002,"5min":0.60599999999999998,"15min":0.40300000000000002},"max":{"1min":1.413,"5min":1.704,"15min":36.295000000000002},"last":1.325},{"interface":"front","average":{"1min":1.208,"5min":1.163,"15min":1.202},"min":{"1min":0.69899999999999995,"5min":0.55300000000000005,"15min":0.42999999999999999},"max":{"1min":1.4830000000000001,"5min":1.706,"15min":3.448},"last":1.4690000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515540,"kb_used_data":214028,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93856300,"statfs":{"total":96636764160,"available":96108851200,"internally_reserved":0,"allocated":219164672,"data_stored":218869425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.76000000000000001,"15min":0.80100000000000005},"min":{"1min":0.36399999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.153,"5min":1.204,"15min":1.901},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81699999999999995,"15min":0.84999999999999998},"min":{"1min":0.378,"5min":0.376,"15min":0.376},"max":{"1min":1.2410000000000001,"5min":1.2509999999999999,"15min":4.0810000000000004},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80100000000000005,"15min":0.81999999999999995},"min":{"1min":0.59899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1950000000000001,"5min":1.429,"15min":2.4809999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.88800000000000001,"15min":0.90600000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1539999999999999,"5min":1.456,"15min":1.7749999999999999},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.91700000000000004,"15min":0.92800000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.41799999999999998},"max":{"1min":1.196,"5min":1.524,"15min":1.774},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.92600000000000005},"min":{"1min":0.47399999999999998,"5min":0.44600000000000001,"15min":0.32500000000000001},"max":{"1min":1.2609999999999999,"5min":1.3160000000000001,"15min":4.1559999999999997},"last":0.87}]},{"osd":3,"last update":"Sun Aug 18 13:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":0.97199999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.36199999999999999},"max":{"1min":1.1719999999999999,"5min":1.4410000000000001,"15min":4.5510000000000002},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.97599999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.52400000000000002},"max":{"1min":1.2350000000000001,"5min":1.4470000000000001,"15min":3.976},"last":0.745}]},{"osd":4,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.96399999999999997,"15min":0.98199999999999998},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.125,"5min":1.5940000000000001,"15min":2.7909999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.081,"5min":1.0780000000000001,"15min":1.103},"min":{"1min":0.71599999999999997,"5min":0.56200000000000006,"15min":0.47899999999999998},"max":{"1min":1.268,"5min":1.6839999999999999,"15min":2.532},"last":0.84499999999999997}]},{"osd":6,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0449999999999999,"15min":1.0820000000000001},"min":{"1min":0.64600000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.421,"5min":1.532,"15min":4.2050000000000001},"last":1.121},{"interface":"front","average":{"1min":0.998,"5min":1.0589999999999999,"15min":1.097},"min":{"1min":0.77000000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.2150000000000001,"5min":1.6479999999999999,"15min":2.911},"last":1.052}]},{"osd":7,"last update":"Sun Aug 18 13:52:46 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.85399999999999998,"5min":0.54500000000000004,"15min":0.54400000000000004},"max":{"1min":1.417,"5min":1.659,"15min":2.569},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.214,"5min":1.163,"15min":1.357},"min":{"1min":0.97499999999999998,"5min":0.50600000000000001,"15min":0.40400000000000003},"max":{"1min":1.4570000000000001,"5min":1.605,"15min":58.290999999999997},"last":1.0860000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":41955328,"data_stored":41943666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50331808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21549056,"data_stored":21547239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":34140160,"data_stored":34130811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":54263808,"data_stored":54247785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41947136,"data_stored":41943130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50339840,"data_stored":50332308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":75227136,"data_stored":75219749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211058688,"data_stored":211058688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334540800,"data_stored":334540800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212602880,"data_stored":212602880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":347811840,"data_stored":347811840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":224706560,"data_stored":224706560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176599040,"data_stored":176599040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":269266944,"data_stored":269266944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":225062912,"data_stored":225062912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T13:53:05.106 DEBUG:teuthology.orchestra.run.smithi118:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-18T13:53:05.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.217+0000 7f0a4bfff700 1 Processor -- start 2024-08-18T13:53:05.222 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4bfff700 1 -- start start 2024-08-18T13:53:05.223 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4bfff700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0a4c0c3b10 0x7f0a4c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a4c0c3ee0 0x7f0a4c067860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4bfff700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c0673d0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4bfff700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c06f7a0 con 0x7f0a4c0c3b10 2024-08-18T13:53:05.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4bfff700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c0bea90 con 0x7f0a4c0c3ee0 2024-08-18T13:53:05.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0a4c0c3b10 0x7f0a4c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.224 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f0a4c0c3b10 0x7f0a4c06a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41288/0 (socket says 172.21.15.118:41288) 2024-08-18T13:53:05.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 -- 172.21.15.118:0/1890473346 learned_addr learned my addr 172.21.15.118:0/1890473346 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:05.225 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4a7fc700 1 --2- 172.21.15.118:0/1890473346 >> v2:172.21.15.178:3300/0 conn(0x7f0a4c0c3ee0 0x7f0a4c067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 -- 172.21.15.118:0/1890473346 >> v2:172.21.15.178:3300/0 conn(0x7f0a4c0c3ee0 msgr2=0x7f0a4c067860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:05.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 --2- 172.21.15.118:0/1890473346 >> v2:172.21.15.178:3300/0 conn(0x7f0a4c0c3ee0 0x7f0a4c067860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 -- 172.21.15.118:0/1890473346 >> v1:172.21.15.178:6789/0 conn(0x7f0a4c06bf70 legacy=0x7f0a4c14cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 -- 172.21.15.118:0/1890473346 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c153190 con 0x7f0a4c0c3b10 2024-08-18T13:53:05.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a4affd700 1 --2- 172.21.15.118:0/1890473346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c0c3b10 0x7f0a4c06a080 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0c37e0 tx=0x7f0a3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5485c064e01ea97e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.221+0000 7f0a49ffb700 1 -- 172.21.15.118:0/1890473346 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c009e70 con 0x7f0a4c0c3b10 2024-08-18T13:53:05.227 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a49ffb700 1 -- 172.21.15.118:0/1890473346 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c00b860 con 0x7f0a4c0c3b10 2024-08-18T13:53:05.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a49ffb700 1 -- 172.21.15.118:0/1890473346 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00bc00 con 0x7f0a4c0c3b10 2024-08-18T13:53:05.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 -- 172.21.15.118:0/1890473346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c0c3b10 msgr2=0x7f0a4c06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/1890473346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c0c3b10 0x7f0a4c06a080 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0c37e0 tx=0x7f0a3c009250 comp rx=0 tx=0).stop 2024-08-18T13:53:05.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 -- 172.21.15.118:0/1890473346 shutdown_connections 2024-08-18T13:53:05.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/1890473346 >> v2:172.21.15.178:3300/0 conn(0x7f0a4c0c3ee0 0x7f0a4c067860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.228 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/1890473346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c0c3b10 0x7f0a4c06a080 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 -- 172.21.15.118:0/1890473346 >> 172.21.15.118:0/1890473346 conn(0x7f0a4c06b1f0 msgr2=0x7f0a4c06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:05.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 -- 172.21.15.118:0/1890473346 shutdown_connections 2024-08-18T13:53:05.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.225+0000 7f0a4bfff700 1 -- 172.21.15.118:0/1890473346 wait complete. 2024-08-18T13:53:05.229 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 Processor -- start 2024-08-18T13:53:05.230 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 -- start start 2024-08-18T13:53:05.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c06bf70 0x7f0a4c156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a4c0c3b10 0x7f0a4c158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.232 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a4c0c3ee0 0x7f0a4c0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c166eb0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c159160 con 0x7f0a4c0c3ee0 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c167100 con 0x7f0a4c0c3b10 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c06bf70 0x7f0a4c156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c06bf70 0x7f0a4c156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41294/0 (socket says 172.21.15.118:41294) 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 -- 172.21.15.118:0/3644222054 learned_addr learned my addr 172.21.15.118:0/3644222054 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4a7fc700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a4c0c3b10 0x7f0a4c158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.233 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4b7fe700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a4c0c3ee0 0x7f0a4c0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 -- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a4c0c3b10 msgr2=0x7f0a4c158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a4c0c3b10 0x7f0a4c158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 -- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a4c0c3ee0 msgr2=0x7f0a4c0c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a4c0c3ee0 0x7f0a4c0c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 -- 172.21.15.118:0/3644222054 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0094f0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.234 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4affd700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c06bf70 0x7f0a4c156f20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c06aea0 tx=0x7f0a3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00bbe0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c011420 con 0x7f0a4c06bf70 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c1673e0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c0117c0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.229+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c1679a0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.233+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0a3c011bf0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.233+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c071410 con 0x7f0a4c06bf70 2024-08-18T13:53:05.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.233+0000 7f0a42ffd700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0a2c060860 0x7f0a2c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.233+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f0a3c07e620 con 0x7f0a4c06bf70 2024-08-18T13:53:05.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.233+0000 7f0a4a7fc700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0a2c060860 0x7f0a2c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.237 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.233+0000 7f0a4a7fc700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0a2c060860 0x7f0a2c062d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0c6ce0 tx=0x7f0a340091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.243 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.241+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0a3c04e4b0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.411 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.409+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0a4c151ea0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.418 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.417+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0a3c057020 con 0x7f0a4c06bf70 2024-08-18T13:53:05.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.417+0000 7f0a42ffd700 1 -- 172.21.15.118:0/3644222054 <== mon.0 v2:172.21.15.118:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0a3c051ad0 con 0x7f0a4c06bf70 2024-08-18T13:53:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0a2c060860 msgr2=0x7f0a2c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0a2c060860 0x7f0a2c062d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0c6ce0 tx=0x7f0a340091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c06bf70 msgr2=0x7f0a4c156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c06bf70 0x7f0a4c156f20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c06aea0 tx=0x7f0a3c017000 comp rx=0 tx=0).stop 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 shutdown_connections 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f0a2c060860 0x7f0a2c062d10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a4c0c3ee0 0x7f0a4c0c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a4c0c3b10 0x7f0a4c158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 --2- 172.21.15.118:0/3644222054 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4c06bf70 0x7f0a4c156f20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 >> 172.21.15.118:0/3644222054 conn(0x7f0a4c06b1f0 msgr2=0x7f0a4c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:05.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 shutdown_connections 2024-08-18T13:53:05.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.421+0000 7f0a4bfff700 1 -- 172.21.15.118:0/3644222054 wait complete. 2024-08-18T13:53:05.437 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-18T13:53:05.437 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:53:05.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef8dd0700 1 Processor -- start 2024-08-18T13:53:05.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef8dd0700 1 -- start start 2024-08-18T13:53:05.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef8dd0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6ef40673d0 0x7f6ef406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef8dd0700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef406ee60 con 0x7f6ef40c8200 2024-08-18T13:53:05.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef8dd0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef4153d10 con 0x7f6ef4153690 2024-08-18T13:53:05.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef8dd0700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef415a640 con 0x7f6ef40673d0 2024-08-18T13:53:05.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef2ffd700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f6ef40c8200 0x7f6ef40c29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53918/0 (socket says 172.21.15.118:53918) 2024-08-18T13:53:05.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef2ffd700 1 -- 172.21.15.118:0/685088941 learned_addr learned my addr 172.21.15.118:0/685088941 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:53:05.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef3fff700 1 --2- 172.21.15.118:0/685088941 >> v2:172.21.15.178:3300/0 conn(0x7f6ef40673d0 0x7f6ef406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef27fc700 1 -- 172.21.15.118:0/685088941 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316345017 0 0) 0x7f6ef406ee60 con 0x7f6ef40c8200 2024-08-18T13:53:05.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef27fc700 1 -- 172.21.15.118:0/685088941 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed4003570 con 0x7f6ef40c8200 2024-08-18T13:53:05.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef27fc700 1 -- 172.21.15.118:0/685088941 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2888209485 0 0) 0x7f6ed4003570 con 0x7f6ef40c8200 2024-08-18T13:53:05.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef3fff700 1 -- 172.21.15.118:0/685088941 >> v1:172.21.15.178:6789/0 conn(0x7f6ef4153690 legacy=0x7f6ef40c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef3fff700 1 -- 172.21.15.118:0/685088941 >> v1:172.21.15.118:6789/0 conn(0x7f6ef40c8200 legacy=0x7f6ef40c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.541+0000 7f6ef3fff700 1 -- 172.21.15.118:0/685088941 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef406f9c0 con 0x7f6ef40673d0 2024-08-18T13:53:05.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef3fff700 1 --2- 172.21.15.118:0/685088941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40673d0 0x7f6ef406f2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40098e0 tx=0x7f6ee40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d10e6790cd53847 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef27fc700 1 -- 172.21.15.118:0/685088941 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0017f0 con 0x7f6ef40673d0 2024-08-18T13:53:05.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef27fc700 1 -- 172.21.15.118:0/685088941 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6ee4011420 con 0x7f6ef40673d0 2024-08-18T13:53:05.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef27fc700 1 -- 172.21.15.118:0/685088941 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40117a0 con 0x7f6ef40673d0 2024-08-18T13:53:05.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/685088941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40673d0 msgr2=0x7f6ef406f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/685088941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40673d0 0x7f6ef406f2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40098e0 tx=0x7f6ee40091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/685088941 shutdown_connections 2024-08-18T13:53:05.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/685088941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40673d0 0x7f6ef406f2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/685088941 >> 172.21.15.118:0/685088941 conn(0x7f6ef406c020 msgr2=0x7f6ef40c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:05.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/685088941 shutdown_connections 2024-08-18T13:53:05.549 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.545+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/685088941 wait complete. 2024-08-18T13:53:05.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 Processor -- start 2024-08-18T13:53:05.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 -- start start 2024-08-18T13:53:05.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ef40673d0 0x7f6ef416d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40c8200 0x7f6ef40c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ef4153690 0x7f6ef40c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4067810 con 0x7f6ef4153690 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef415bb80 con 0x7f6ef40c8200 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef8dd0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4068d60 con 0x7f6ef40673d0 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ef4153690 0x7f6ef40c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ef4153690 0x7f6ef40c6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41296/0 (socket says 172.21.15.118:41296) 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 -- 172.21.15.118:0/517444330 learned_addr learned my addr 172.21.15.118:0/517444330 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:05.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef2ffd700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40c8200 0x7f6ef40c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef37fe700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ef40673d0 0x7f6ef416d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 -- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ef40673d0 msgr2=0x7f6ef416d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ef40673d0 0x7f6ef416d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 -- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40c8200 msgr2=0x7f6ef40c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40c8200 0x7f6ef40c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.549+0000 7f6ef3fff700 1 -- 172.21.15.118:0/517444330 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4009420 con 0x7f6ef4153690 2024-08-18T13:53:05.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6ef3fff700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ef4153690 0x7f6ef40c6ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6ef405ebf0 tx=0x7f6ee40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6eeb7fe700 1 -- 172.21.15.118:0/517444330 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee401cc00 con 0x7f6ef4153690 2024-08-18T13:53:05.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6eeb7fe700 1 -- 172.21.15.118:0/517444330 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6ee401cd60 con 0x7f6ef4153690 2024-08-18T13:53:05.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef4068750 con 0x7f6ef4153690 2024-08-18T13:53:05.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6eeb7fe700 1 -- 172.21.15.118:0/517444330 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee401a6c0 con 0x7f6ef4153690 2024-08-18T13:53:05.556 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef415a8f0 con 0x7f6ef4153690 2024-08-18T13:53:05.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef40c6680 con 0x7f6ef4153690 2024-08-18T13:53:05.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.553+0000 7f6eeb7fe700 1 -- 172.21.15.118:0/517444330 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6ee401aa30 con 0x7f6ef4153690 2024-08-18T13:53:05.557 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.557+0000 7f6eeb7fe700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ed8060860 0x7f6ed8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.557+0000 7f6eeb7fe700 1 -- 172.21.15.118:0/517444330 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f6ee407ec70 con 0x7f6ef4153690 2024-08-18T13:53:05.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.557+0000 7f6ef37fe700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ed8060860 0x7f6ed8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.558 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.557+0000 7f6ef37fe700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ed8060860 0x7f6ed8062d10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6edc003a80 tx=0x7f6edc007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.562 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.561+0000 7f6eeb7fe700 1 -- 172.21.15.118:0/517444330 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6ef40c6680 con 0x7f6ef4153690 2024-08-18T13:53:05.760 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.757+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ef40c6680 con 0x7f6ef4153690 2024-08-18T13:53:05.761 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6eeb7fe700 1 -- 172.21.15.118:0/517444330 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f6ef40c6680 con 0x7f6ef4153690 2024-08-18T13:53:05.762 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:53:05.762 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5752,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2141281823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2141281823},{"type":"v1","addr":"172.21.15.178:6835","nonce":2141281823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:18.658279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":22,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:53:05.764 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ed8060860 msgr2=0x7f6ed8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.764 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ed8060860 0x7f6ed8062d10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6edc003a80 tx=0x7f6edc007ec0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.764 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ef4153690 msgr2=0x7f6ef40c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.764 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ef4153690 0x7f6ef40c6ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6ef405ebf0 tx=0x7f6ee40091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 shutdown_connections 2024-08-18T13:53:05.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f6ed8060860 0x7f6ed8062d10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ef4153690 0x7f6ef40c6ea0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ef40c8200 0x7f6ef40c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 --2- 172.21.15.118:0/517444330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ef40673d0 0x7f6ef416d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 >> 172.21.15.118:0/517444330 conn(0x7f6ef406c020 msgr2=0x7f6ef406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:05.765 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 shutdown_connections 2024-08-18T13:53:05.766 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.761+0000 7f6ef8dd0700 1 -- 172.21.15.118:0/517444330 wait complete. 2024-08-18T13:53:05.766 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 42 2024-08-18T13:53:05.776 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-18T13:53:05.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2be4922700 1 Processor -- start 2024-08-18T13:53:05.886 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2be4922700 1 -- start start 2024-08-18T13:53:05.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2be4922700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2be014f410 0x7f2be00ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.887 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2be4922700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f7e0 0x7f2be00abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2be4922700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be00c1c80 con 0x7f2be0158830 2024-08-18T13:53:05.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2be4922700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be00c18e0 con 0x7f2be014f410 2024-08-18T13:53:05.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2be4922700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00c09f0 con 0x7f2be014f7e0 2024-08-18T13:53:05.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bdf59e700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2be0158830 0x7f2be00c7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53934/0 (socket says 172.21.15.118:53934) 2024-08-18T13:53:05.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bde59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f7e0 0x7f2be00abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bdf59e700 1 -- 172.21.15.118:0/1755934905 learned_addr learned my addr 172.21.15.118:0/1755934905 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:53:05.888 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bded9d700 1 --2- 172.21.15.118:0/1755934905 >> v2:172.21.15.178:3300/0 conn(0x7f2be014f410 0x7f2be00ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.890 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bddd9b700 1 -- 172.21.15.118:0/1755934905 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459012685 0 0) 0x7f2be00c1c80 con 0x7f2be0158830 2024-08-18T13:53:05.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bddd9b700 1 -- 172.21.15.118:0/1755934905 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bcc003570 con 0x7f2be0158830 2024-08-18T13:53:05.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bded9d700 1 -- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f7e0 msgr2=0x7f2be00abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:05.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bded9d700 1 --2- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f7e0 0x7f2be00abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bded9d700 1 -- 172.21.15.118:0/1755934905 >> v1:172.21.15.118:6789/0 conn(0x7f2be0158830 legacy=0x7f2be00c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bded9d700 1 -- 172.21.15.118:0/1755934905 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00c0cd0 con 0x7f2be014f410 2024-08-18T13:53:05.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bde59c700 1 --2- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f7e0 0x7f2be00abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:53:05.891 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bded9d700 1 --2- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be014f410 0x7f2be00ab790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009910 tx=0x7f2bd00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edcc773c5e2546e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.885+0000 7f2bddd9b700 1 -- 172.21.15.118:0/1755934905 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0011630 con 0x7f2be014f410 2024-08-18T13:53:05.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2bddd9b700 1 -- 172.21.15.118:0/1755934905 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2bd0011790 con 0x7f2be014f410 2024-08-18T13:53:05.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2bddd9b700 1 -- 172.21.15.118:0/1755934905 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0011b50 con 0x7f2be014f410 2024-08-18T13:53:05.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 -- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be014f410 msgr2=0x7f2be00ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 --2- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be014f410 0x7f2be00ab790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009910 tx=0x7f2bd00091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 -- 172.21.15.118:0/1755934905 shutdown_connections 2024-08-18T13:53:05.892 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 --2- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f7e0 0x7f2be00abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 --2- 172.21.15.118:0/1755934905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be014f410 0x7f2be00ab790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 -- 172.21.15.118:0/1755934905 >> 172.21.15.118:0/1755934905 conn(0x7f2be00c8620 msgr2=0x7f2be00c8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:05.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 -- 172.21.15.118:0/1755934905 shutdown_connections 2024-08-18T13:53:05.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 -- 172.21.15.118:0/1755934905 wait complete. 2024-08-18T13:53:05.893 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.889+0000 7f2be4922700 1 Processor -- start 2024-08-18T13:53:05.894 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 -- start start 2024-08-18T13:53:05.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f410 0x7f2be01559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be014f7e0 0x7f2be00c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be0158830 0x7f2be00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be0155100 con 0x7f2be014f7e0 2024-08-18T13:53:05.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00c77c0 con 0x7f2be0158830 2024-08-18T13:53:05.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00c8350 con 0x7f2be014f410 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be014f7e0 0x7f2be00c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be014f7e0 0x7f2be00c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41298/0 (socket says 172.21.15.118:41298) 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 -- 172.21.15.118:0/193223805 learned_addr learned my addr 172.21.15.118:0/193223805 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bded9d700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f410 0x7f2be01559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bdf59e700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be0158830 0x7f2be00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 -- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f410 msgr2=0x7f2be01559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f410 0x7f2be01559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.897 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 -- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be0158830 msgr2=0x7f2be00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:05.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be0158830 0x7f2be00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:05.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 -- 172.21.15.118:0/193223805 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc8009420 con 0x7f2be014f7e0 2024-08-18T13:53:05.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bde59c700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be014f7e0 0x7f2be00c9990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ec80 tx=0x7f2bc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bd6ffd700 1 -- 172.21.15.118:0/193223805 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800c850 con 0x7f2be014f7e0 2024-08-18T13:53:05.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bd6ffd700 1 -- 172.21.15.118:0/193223805 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2bc800c9b0 con 0x7f2be014f7e0 2024-08-18T13:53:05.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2bd6ffd700 1 -- 172.21.15.118:0/193223805 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800cd70 con 0x7f2be014f7e0 2024-08-18T13:53:05.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd0009420 con 0x7f2be014f7e0 2024-08-18T13:53:05.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.893+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be00c8090 con 0x7f2be014f7e0 2024-08-18T13:53:05.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.897+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be00713a0 con 0x7f2be014f7e0 2024-08-18T13:53:05.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.897+0000 7f2bd6ffd700 1 -- 172.21.15.118:0/193223805 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2bc8013680 con 0x7f2be014f7e0 2024-08-18T13:53:05.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.897+0000 7f2bd6ffd700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bc0060860 0x7f2bc0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:05.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.897+0000 7f2bd6ffd700 1 -- 172.21.15.118:0/193223805 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f2bc807d830 con 0x7f2be014f7e0 2024-08-18T13:53:05.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.897+0000 7f2bded9d700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bc0060860 0x7f2bc0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:05.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.897+0000 7f2bded9d700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bc0060860 0x7f2bc0062d10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0003080 tx=0x7f2bd00091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:05.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:05.905+0000 7f2bd6ffd700 1 -- 172.21.15.118:0/193223805 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2bc804d740 con 0x7f2be014f7e0 2024-08-18T13:53:06.108 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.105+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2be0153120 con 0x7f2be014f7e0 2024-08-18T13:53:06.109 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.109+0000 7f2bd6ffd700 1 -- 172.21.15.118:0/193223805 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+6130 (secure 0 0 0) 0x7f2bc8050d60 con 0x7f2be014f7e0 2024-08-18T13:53:06.110 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:53:06.110 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3872383872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3872383872},{"type":"v1","addr":"172.21.15.118:6835","nonce":3872383872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/1680290813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":1680290813},{"type":"v1","addr":"172.21.15.118:6837","nonce":1680290813}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4296,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/2444565967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":2444565967},{"type":"v1","addr":"172.21.15.118:6839","nonce":2444565967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/4030958675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4030958675},{"type":"v1","addr":"172.21.15.178:6834","nonce":4030958675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5752,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/2141281823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2141281823},{"type":"v1","addr":"172.21.15.178:6835","nonce":2141281823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T13:39:34.735047+0000","modified":"2024-08-18T13:51:18.658279+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5195":{"gid":5195,"name":"d","rank":0,"incarnation":37,"state":"up:active","state_seq":22,"addr":"172.21.15.178:6837/3317475884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3317475884},{"type":"v1","addr":"172.21.15.178:6837","nonce":3317475884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-18T13:53:06.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.109+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bc0060860 msgr2=0x7f2bc0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.109+0000 7f2be4922700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bc0060860 0x7f2bc0062d10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0003080 tx=0x7f2bd00091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.109+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be014f7e0 msgr2=0x7f2be00c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be014f7e0 0x7f2be00c9990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ec80 tx=0x7f2bc80091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 shutdown_connections 2024-08-18T13:53:06.114 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2bc0060860 0x7f2bc0062d10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be0158830 0x7f2be00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2be014f7e0 0x7f2be00c9990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 --2- 172.21.15.118:0/193223805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be014f410 0x7f2be01559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 >> 172.21.15.118:0/193223805 conn(0x7f2be00c8620 msgr2=0x7f2be00c1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:06.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 shutdown_connections 2024-08-18T13:53:06.115 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.113+0000 7f2be4922700 1 -- 172.21.15.118:0/193223805 wait complete. 2024-08-18T13:53:06.116 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 42 2024-08-18T13:53:06.130 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T13:53:06.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff759e700 1 Processor -- start 2024-08-18T13:53:06.236 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff759e700 1 -- start start 2024-08-18T13:53:06.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff759e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2ff8153690 0x7f2ff80673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff759e700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff806ee60 con 0x7f2ff806f2f0 2024-08-18T13:53:06.238 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff759e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff806f730 con 0x7f2ff80c8200 2024-08-18T13:53:06.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff759e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff815a640 con 0x7f2ff8153690 2024-08-18T13:53:06.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff6d9d700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f2ff806f2f0 0x7f2ff80c29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53944/0 (socket says 172.21.15.118:53944) 2024-08-18T13:53:06.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff659c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f2ff8153690 0x7f2ff80673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff6d9d700 1 -- 172.21.15.118:0/406037165 learned_addr learned my addr 172.21.15.118:0/406037165 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:53:06.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.233+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008928532 0 0) 0x7f2ff806ee60 con 0x7f2ff806f2f0 2024-08-18T13:53:06.239 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd8003570 con 0x7f2ff806f2f0 2024-08-18T13:53:06.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733085414 0 0) 0x7f2ff806f730 con 0x7f2ff80c8200 2024-08-18T13:53:06.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff806ee60 con 0x7f2ff80c8200 2024-08-18T13:53:06.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431314755 0 0) 0x7f2fd8003570 con 0x7f2ff806f2f0 2024-08-18T13:53:06.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff659c700 1 -- 172.21.15.118:0/406037165 >> v1:172.21.15.178:6789/0 conn(0x7f2ff80c8200 legacy=0x7f2ff80c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff659c700 1 -- 172.21.15.118:0/406037165 >> v1:172.21.15.118:6789/0 conn(0x7f2ff806f2f0 legacy=0x7f2ff80c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff659c700 1 -- 172.21.15.118:0/406037165 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8153d10 con 0x7f2ff8153690 2024-08-18T13:53:06.240 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff659c700 1 --2- 172.21.15.118:0/406037165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff8153690 0x7f2ff80673d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80098e0 tx=0x7f2fe80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29d58f1a6020eb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00017f0 con 0x7f2ff8153690 2024-08-18T13:53:06.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2fe8011420 con 0x7f2ff8153690 2024-08-18T13:53:06.241 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff559a700 1 -- 172.21.15.118:0/406037165 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80117a0 con 0x7f2ff8153690 2024-08-18T13:53:06.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 -- 172.21.15.118:0/406037165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff8153690 msgr2=0x7f2ff80673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.244 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 --2- 172.21.15.118:0/406037165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff8153690 0x7f2ff80673d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80098e0 tx=0x7f2fe80091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 -- 172.21.15.118:0/406037165 shutdown_connections 2024-08-18T13:53:06.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 --2- 172.21.15.118:0/406037165 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff8153690 0x7f2ff80673d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 -- 172.21.15.118:0/406037165 >> 172.21.15.118:0/406037165 conn(0x7f2ff806c020 msgr2=0x7f2ff80c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:06.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 -- 172.21.15.118:0/406037165 shutdown_connections 2024-08-18T13:53:06.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 -- 172.21.15.118:0/406037165 wait complete. 2024-08-18T13:53:06.245 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 Processor -- start 2024-08-18T13:53:06.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.237+0000 7f2ff759e700 1 -- start start 2024-08-18T13:53:06.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ff806f2f0 0x7f2ff8156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff80c8200 0x7f2ff80c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ff8153690 0x7f2ff814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff816a690 con 0x7f2ff80c8200 2024-08-18T13:53:06.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815c470 con 0x7f2ff806f2f0 2024-08-18T13:53:06.246 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8169910 con 0x7f2ff8153690 2024-08-18T13:53:06.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff80c8200 0x7f2ff80c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff80c8200 0x7f2ff80c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41308/0 (socket says 172.21.15.118:41308) 2024-08-18T13:53:06.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ff806f2f0 0x7f2ff8156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 -- 172.21.15.118:0/2471650375 learned_addr learned my addr 172.21.15.118:0/2471650375 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:06.247 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff6d9d700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ff8153690 0x7f2ff814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 -- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ff8153690 msgr2=0x7f2ff814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ff8153690 0x7f2ff814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 -- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ff806f2f0 msgr2=0x7f2ff8156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ff806f2f0 0x7f2ff8156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 -- 172.21.15.118:0/2471650375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8009420 con 0x7f2ff80c8200 2024-08-18T13:53:06.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff6d9d700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ff8153690 0x7f2ff814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:53:06.248 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff5d9b700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff80c8200 0x7f2ff80c6680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c7c0 tx=0x7f2fe0007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2fee7fc700 1 -- 172.21.15.118:0/2471650375 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0010510 con 0x7f2ff80c8200 2024-08-18T13:53:06.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2fee7fc700 1 -- 172.21.15.118:0/2471650375 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2fe0010670 con 0x7f2ff80c8200 2024-08-18T13:53:06.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff815c190 con 0x7f2ff80c8200 2024-08-18T13:53:06.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2fee7fc700 1 -- 172.21.15.118:0/2471650375 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0010a50 con 0x7f2ff80c8200 2024-08-18T13:53:06.249 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff8169e90 con 0x7f2ff80c8200 2024-08-18T13:53:06.250 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.241+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff80713a0 con 0x7f2ff80c8200 2024-08-18T13:53:06.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.245+0000 7f2fee7fc700 1 -- 172.21.15.118:0/2471650375 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2fe001c440 con 0x7f2ff80c8200 2024-08-18T13:53:06.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.245+0000 7f2fee7fc700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2fdc060860 0x7f2fdc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.245+0000 7f2ff659c700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2fdc060860 0x7f2fdc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.245+0000 7f2fee7fc700 1 -- 172.21.15.118:0/2471650375 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7f2fe007c9c0 con 0x7f2ff80c8200 2024-08-18T13:53:06.251 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.245+0000 7f2ff659c700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2fdc060860 0x7f2fdc062d10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80093f0 tx=0x7f2fe80091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.254 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.253+0000 7f2fee7fc700 1 -- 172.21.15.118:0/2471650375 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2fe004c830 con 0x7f2ff80c8200 2024-08-18T13:53:06.419 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.416+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ff80c7fc0 con 0x7f2ff80c8200 2024-08-18T13:53:06.421 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.416+0000 7f2fee7fc700 1 -- 172.21.15.118:0/2471650375 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18405 (secure 0 0 0) 0x7f2fe004fe50 con 0x7f2ff80c8200 2024-08-18T13:53:06.421 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-08-18T13:53:06.422 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":54,"fsid":"ac1e0c0e-66a6-4fa9-9612-9b35e09366c7","created":"2024-08-18T13:39:10.731657+0000","modified":"2024-08-18T13:50:50.639291+0000","last_up_change":"2024-08-18T13:39:21.399940+0000","last_in_change":"2024-08-18T13:39:12.787115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T13:39:21.246221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T13:39:32.826757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T13:39:33.731815+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T13:39:37.447514+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6faa90ce-e1a9-481a-8f60-55bc2ed16004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":24281},{"type":"v1","addr":"172.21.15.118:6825","nonce":24281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":24281},{"type":"v1","addr":"172.21.15.118:6827","nonce":24281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":24281},{"type":"v1","addr":"172.21.15.118:6831","nonce":24281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":24281},{"type":"v1","addr":"172.21.15.118:6829","nonce":24281}]},"public_addr":"172.21.15.118:6825/24281","cluster_addr":"172.21.15.118:6827/24281","heartbeat_back_addr":"172.21.15.118:6831/24281","heartbeat_front_addr":"172.21.15.118:6829/24281","state":["exists","up"]},{"osd":1,"uuid":"9809a5fe-2ee4-4e13-95b8-35e09a43fa61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":24279},{"type":"v1","addr":"172.21.15.118:6817","nonce":24279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":24279},{"type":"v1","addr":"172.21.15.118:6819","nonce":24279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":24279},{"type":"v1","addr":"172.21.15.118:6823","nonce":24279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":24279},{"type":"v1","addr":"172.21.15.118:6821","nonce":24279}]},"public_addr":"172.21.15.118:6817/24279","cluster_addr":"172.21.15.118:6819/24279","heartbeat_back_addr":"172.21.15.118:6823/24279","heartbeat_front_addr":"172.21.15.118:6821/24279","state":["exists","up"]},{"osd":2,"uuid":"d1dea5ed-5651-486c-99fa-f82ca3216928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":24282},{"type":"v1","addr":"172.21.15.118:6801","nonce":24282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6803","nonce":24282},{"type":"v1","addr":"172.21.15.118:6805","nonce":24282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6811","nonce":24282},{"type":"v1","addr":"172.21.15.118:6813","nonce":24282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6807","nonce":24282},{"type":"v1","addr":"172.21.15.118:6809","nonce":24282}]},"public_addr":"172.21.15.118:6801/24282","cluster_addr":"172.21.15.118:6805/24282","heartbeat_back_addr":"172.21.15.118:6813/24282","heartbeat_front_addr":"172.21.15.118:6809/24282","state":["exists","up"]},{"osd":3,"uuid":"0692e5ad-4c32-4a91-a7d0-a0c7b5bfc0de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":24280},{"type":"v1","addr":"172.21.15.118:6804","nonce":24280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":24280},{"type":"v1","addr":"172.21.15.118:6808","nonce":24280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":24280},{"type":"v1","addr":"172.21.15.118:6815","nonce":24280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":24280},{"type":"v1","addr":"172.21.15.118:6812","nonce":24280}]},"public_addr":"172.21.15.118:6804/24280","cluster_addr":"172.21.15.118:6808/24280","heartbeat_back_addr":"172.21.15.118:6815/24280","heartbeat_front_addr":"172.21.15.118:6812/24280","state":["exists","up"]},{"osd":4,"uuid":"66ab9cf2-8285-49a5-b2ed-c3931c19a983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":24441},{"type":"v1","addr":"172.21.15.178:6809","nonce":24441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":24441},{"type":"v1","addr":"172.21.15.178:6811","nonce":24441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":24441},{"type":"v1","addr":"172.21.15.178:6815","nonce":24441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":24441},{"type":"v1","addr":"172.21.15.178:6813","nonce":24441}]},"public_addr":"172.21.15.178:6809/24441","cluster_addr":"172.21.15.178:6811/24441","heartbeat_back_addr":"172.21.15.178:6815/24441","heartbeat_front_addr":"172.21.15.178:6813/24441","state":["exists","up"]},{"osd":5,"uuid":"45763a0e-ecdc-4810-bc05-f1a2f14f2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":24442},{"type":"v1","addr":"172.21.15.178:6820","nonce":24442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":24442},{"type":"v1","addr":"172.21.15.178:6826","nonce":24442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":24442},{"type":"v1","addr":"172.21.15.178:6831","nonce":24442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":24442},{"type":"v1","addr":"172.21.15.178:6829","nonce":24442}]},"public_addr":"172.21.15.178:6820/24442","cluster_addr":"172.21.15.178:6826/24442","heartbeat_back_addr":"172.21.15.178:6831/24442","heartbeat_front_addr":"172.21.15.178:6829/24442","state":["exists","up"]},{"osd":6,"uuid":"e5380e12-4fba-4df2-9efb-c0e472e52c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":24448},{"type":"v1","addr":"172.21.15.178:6818","nonce":24448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":24448},{"type":"v1","addr":"172.21.15.178:6821","nonce":24448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":24448},{"type":"v1","addr":"172.21.15.178:6827","nonce":24448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":24448},{"type":"v1","addr":"172.21.15.178:6824","nonce":24448}]},"public_addr":"172.21.15.178:6818/24448","cluster_addr":"172.21.15.178:6821/24448","heartbeat_back_addr":"172.21.15.178:6827/24448","heartbeat_front_addr":"172.21.15.178:6824/24448","state":["exists","up"]},{"osd":7,"uuid":"8ffff12a-8dde-4ae8-8b1e-14e43b435b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":24440},{"type":"v1","addr":"172.21.15.178:6801","nonce":24440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":24440},{"type":"v1","addr":"172.21.15.178:6803","nonce":24440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":24440},{"type":"v1","addr":"172.21.15.178:6807","nonce":24440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":24440},{"type":"v1","addr":"172.21.15.178:6805","nonce":24440}]},"public_addr":"172.21.15.178:6801/24440","cluster_addr":"172.21.15.178:6803/24440","heartbeat_back_addr":"172.21.15.178:6807/24440","heartbeat_front_addr":"172.21.15.178:6805/24440","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.011838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.129616+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.967771+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:17.867458+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.296219+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.070263+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.304323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T13:39:18.171947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6835/2054758182":"2024-08-19T13:50:50.639112+0000","172.21.15.178:6836/1054354354":"2024-08-19T13:49:30.622514+0000","172.21.15.178:6837/3452743394":"2024-08-19T13:45:35.570038+0000","172.21.15.178:6832/679522662":"2024-08-19T13:47:20.593054+0000","172.21.15.178:6836/3452743394":"2024-08-19T13:45:35.570038+0000","172.21.15.178:6832/1590569622":"2024-08-19T13:44:15.544758+0000","172.21.15.178:6832/2054758182":"2024-08-19T13:50:50.639112+0000","172.21.15.178:6837/1054354354":"2024-08-19T13:49:30.622514+0000","172.21.15.178:6835/679522662":"2024-08-19T13:47:20.593054+0000","172.21.15.178:6835/1590569622":"2024-08-19T13:44:15.544758+0000","172.21.15.178:6837/2503344632":"2024-08-19T13:42:15.510903+0000","172.21.15.178:6836/2503344632":"2024-08-19T13:42:15.510903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T13:53:06.423 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2fdc060860 msgr2=0x7f2fdc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2fdc060860 0x7f2fdc062d10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80093f0 tx=0x7f2fe80091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff80c8200 msgr2=0x7f2ff80c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff80c8200 0x7f2ff80c6680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c7c0 tx=0x7f2fe0007dc0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 shutdown_connections 2024-08-18T13:53:06.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f2fdc060860 0x7f2fdc062d10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.424 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ff8153690 0x7f2ff814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2ff80c8200 0x7f2ff80c6680 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 --2- 172.21.15.118:0/2471650375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ff806f2f0 0x7f2ff8156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 >> 172.21.15.118:0/2471650375 conn(0x7f2ff806c020 msgr2=0x7f2ff80c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:06.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 shutdown_connections 2024-08-18T13:53:06.425 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.420+0000 7f2ff759e700 1 -- 172.21.15.118:0/2471650375 wait complete. 2024-08-18T13:53:06.436 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-18T13:53:06.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81fb50700 1 Processor -- start 2024-08-18T13:53:06.538 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81fb50700 1 -- start start 2024-08-18T13:53:06.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81fb50700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc81813e840 0x7fc8180673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.539 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81fb50700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 0x7fc81806a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81fb50700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc818156040 con 0x7fc818155510 2024-08-18T13:53:06.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81fb50700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81806e940 con 0x7fc81813e840 2024-08-18T13:53:06.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81fb50700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806f4c0 con 0x7fc81806a310 2024-08-18T13:53:06.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81e0ed700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7fc818155510 0x7fc818155930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53958/0 (socket says 172.21.15.118:53958) 2024-08-18T13:53:06.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81e0ed700 1 -- 172.21.15.118:0/3387105617 learned_addr learned my addr 172.21.15.118:0/3387105617 (peer_addr_for_me v1:172.21.15.118:0/0) 2024-08-18T13:53:06.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81d8ec700 1 --2- 172.21.15.118:0/3387105617 >> v2:172.21.15.178:3300/0 conn(0x7fc81813e840 0x7fc8180673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.540 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81d0eb700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 0x7fc81806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.543 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 <== mon.0 v1:172.21.15.118:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921975730 0 0) 0x7fc810001490 con 0x7fc818155510 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 --> v1:172.21.15.118:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc804003570 con 0x7fc818155510 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 <== mon.0 v1:172.21.15.118:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1519948022 0 0) 0x7fc804003570 con 0x7fc818155510 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 msgr2=0x7fc81806a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 0x7fc81806a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 >> v2:172.21.15.178:3300/0 conn(0x7fc81813e840 msgr2=0x7fc8180673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 --2- 172.21.15.118:0/3387105617 >> v2:172.21.15.178:3300/0 conn(0x7fc81813e840 0x7fc8180673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 --> v1:172.21.15.118:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818156280 con 0x7fc818155510 2024-08-18T13:53:06.544 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 <== mon.0 v1:172.21.15.118:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2032032087 0 0) 0x7fc810001980 con 0x7fc818155510 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 >> v1:172.21.15.118:6789/0 conn(0x7fc818155510 legacy=0x7fc818155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 0x7fc804003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc804003fe0 0x7fc804004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc804004d50 0x7fc8040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806e940 con 0x7fc804004d50 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806f4c0 con 0x7fc804003fe0 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80400ba90 con 0x7fc81806a310 2024-08-18T13:53:06.545 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81e0ed700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc804004d50 0x7fc8040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81d0eb700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc804003fe0 0x7fc804004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.536+0000 7fc81d8ec700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 0x7fc804003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81e0ed700 1 -- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 msgr2=0x7fc804003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:06.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81e0ed700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc81806a310 0x7fc804003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81e0ed700 1 -- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc804003fe0 msgr2=0x7fc804004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:06.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81e0ed700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc804003fe0 0x7fc804004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.546 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81e0ed700 1 -- 172.21.15.118:0/3387105617 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc804003570 con 0x7fc804004d50 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81e0ed700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc804004d50 0x7fc8040051c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc81000aaf0 tx=0x7fc8180bcd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1781682483b321d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000a480 con 0x7fc804004d50 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc81000c570 con 0x7fc804004d50 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81c8ea700 1 -- 172.21.15.118:0/3387105617 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000c950 con 0x7fc804004d50 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81fb50700 1 -- 172.21.15.118:0/3387105617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc804004d50 msgr2=0x7fc8040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81fb50700 1 --2- 172.21.15.118:0/3387105617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc804004d50 0x7fc8040051c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc81000aaf0 tx=0x7fc8180bcd70 comp rx=0 tx=0).stop 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81e0ed700 1 -- 172.21.15.118:0/3387105617 reap_dead start 2024-08-18T13:53:06.547 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81fb50700 1 -- 172.21.15.118:0/3387105617 shutdown_connections 2024-08-18T13:53:06.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81fb50700 1 -- 172.21.15.118:0/3387105617 >> 172.21.15.118:0/3387105617 conn(0x7fc8180c8660 msgr2=0x7fc8180c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:06.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81fb50700 1 -- 172.21.15.118:0/3387105617 shutdown_connections 2024-08-18T13:53:06.548 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.540+0000 7fc81fb50700 1 -- 172.21.15.118:0/3387105617 wait complete. 2024-08-18T13:53:06.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 Processor -- start 2024-08-18T13:53:06.550 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 -- start start 2024-08-18T13:53:06.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8181582c0 0x7fc8181657b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc818158c40 0x7fc818165cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc818158690 0x7fc81813a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818150430 con 0x7fc818158690 2024-08-18T13:53:06.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818166230 con 0x7fc8181582c0 2024-08-18T13:53:06.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81fb50700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81813d250 con 0x7fc818158c40 2024-08-18T13:53:06.551 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc818158690 0x7fc81813a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc818158690 0x7fc81813a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41314/0 (socket says 172.21.15.118:41314) 2024-08-18T13:53:06.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 -- 172.21.15.118:0/4288630579 learned_addr learned my addr 172.21.15.118:0/4288630579 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:06.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81d8ec700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8181582c0 0x7fc8181657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81d0eb700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc818158c40 0x7fc818165cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 -- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc818158c40 msgr2=0x7fc818165cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc818158c40 0x7fc818165cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.552 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 -- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8181582c0 msgr2=0x7fc8181657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8181582c0 0x7fc8181657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.544+0000 7fc81e0ed700 1 -- 172.21.15.118:0/4288630579 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818153f70 con 0x7fc818158690 2024-08-18T13:53:06.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc81e0ed700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc818158690 0x7fc81813a330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc810012000 tx=0x7fc8180bcd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc809ffb700 1 -- 172.21.15.118:0/4288630579 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810001940 con 0x7fc818158690 2024-08-18T13:53:06.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc809ffb700 1 -- 172.21.15.118:0/4288630579 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc810021070 con 0x7fc818158690 2024-08-18T13:53:06.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc818159180 con 0x7fc818158690 2024-08-18T13:53:06.553 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc809ffb700 1 -- 172.21.15.118:0/4288630579 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810026430 con 0x7fc818158690 2024-08-18T13:53:06.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818154650 con 0x7fc818158690 2024-08-18T13:53:06.554 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8180642f0 con 0x7fc818158690 2024-08-18T13:53:06.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc809ffb700 1 -- 172.21.15.118:0/4288630579 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc810023030 con 0x7fc818158690 2024-08-18T13:53:06.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc809ffb700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc7fc060860 0x7fc7fc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc809ffb700 1 -- 172.21.15.118:0/4288630579 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7175+0+0 (secure 0 0 0) 0x7fc81007efb0 con 0x7fc818158690 2024-08-18T13:53:06.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.548+0000 7fc81d8ec700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc7fc060860 0x7fc7fc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.555 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.552+0000 7fc81d8ec700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc7fc060860 0x7fc7fc062d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc814006f00 tx=0x7fc8140091a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.559 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.556+0000 7fc809ffb700 1 -- 172.21.15.118:0/4288630579 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc8100af480 con 0x7fc818158690 2024-08-18T13:53:06.764 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.760+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc818156040 con 0x7fc818158690 2024-08-18T13:53:06.775 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.772+0000 7fc809ffb700 1 -- 172.21.15.118:0/4288630579 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc818156040 con 0x7fc818158690 2024-08-18T13:53:06.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc7fc060860 msgr2=0x7fc7fc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc7fc060860 0x7fc7fc062d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc814006f00 tx=0x7fc8140091a0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc818158690 msgr2=0x7fc81813a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc818158690 0x7fc81813a330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc810012000 tx=0x7fc8180bcd70 comp rx=0 tx=0).stop 2024-08-18T13:53:06.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 shutdown_connections 2024-08-18T13:53:06.779 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7fc7fc060860 0x7fc7fc062d10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc818158690 0x7fc81813a330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc818158c40 0x7fc818165cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 --2- 172.21.15.118:0/4288630579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8181582c0 0x7fc8181657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 >> 172.21.15.118:0/4288630579 conn(0x7fc8180c8660 msgr2=0x7fc81806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:06.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 shutdown_connections 2024-08-18T13:53:06.780 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.776+0000 7fc81fb50700 1 -- 172.21.15.118:0/4288630579 wait complete. 2024-08-18T13:53:06.781 INFO:teuthology.orchestra.run.smithi118.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-18T13:53:06.790 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-18T13:53:06.816 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-08-18T13:53:06.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.892+0000 7f13324dc700 1 Processor -- start 2024-08-18T13:53:06.896 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.892+0000 7f13324dc700 1 -- start start 2024-08-18T13:53:06.898 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f132c0bf8e0 0x7f132c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- --> v1:172.21.15.118:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132c156130 con 0x7f132c14c030 2024-08-18T13:53:06.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132c06d070 con 0x7f132c14c400 2024-08-18T13:53:06.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132c155560 con 0x7f132c0bf8e0 2024-08-18T13:53:06.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f1331cdb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f132c0bf8e0 0x7f132c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f1331cdb700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7f132c0bf8e0 0x7f132c0c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41326/0 (socket says 172.21.15.118:41326) 2024-08-18T13:53:06.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13314da700 1 --1- >> v1:172.21.15.118:6789/0 conn(0x7f132c14c030 0x7f132c0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.118:6789/0 says I am v1:172.21.15.118:53968/0 (socket says 172.21.15.118:53968) 2024-08-18T13:53:06.899 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f1331cdb700 1 -- 172.21.15.118:0/3052908603 learned_addr learned my addr 172.21.15.118:0/3052908603 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:06.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f1331cdb700 1 -- 172.21.15.118:0/3052908603 >> v1:172.21.15.178:6789/0 conn(0x7f132c14c400 legacy=0x7f132c06c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f1331cdb700 1 -- 172.21.15.118:0/3052908603 >> v1:172.21.15.118:6789/0 conn(0x7f132c14c030 legacy=0x7f132c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f1331cdb700 1 -- 172.21.15.118:0/3052908603 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c0c9890 con 0x7f132c0bf8e0 2024-08-18T13:53:06.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f1331cdb700 1 --2- 172.21.15.118:0/3052908603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c0bf8e0 0x7f132c0c42d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1324009f20 tx=0x7f1324009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a693485f87c60a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f131ffff700 1 -- 172.21.15.118:0/3052908603 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132400e070 con 0x7f132c0bf8e0 2024-08-18T13:53:06.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f131ffff700 1 -- 172.21.15.118:0/3052908603 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f132400bb10 con 0x7f132c0bf8e0 2024-08-18T13:53:06.900 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f131ffff700 1 -- 172.21.15.118:0/3052908603 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240092e0 con 0x7f132c0bf8e0 2024-08-18T13:53:06.901 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- 172.21.15.118:0/3052908603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c0bf8e0 msgr2=0x7f132c0c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 --2- 172.21.15.118:0/3052908603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c0bf8e0 0x7f132c0c42d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1324009f20 tx=0x7f1324009840 comp rx=0 tx=0).stop 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- 172.21.15.118:0/3052908603 shutdown_connections 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 --2- 172.21.15.118:0/3052908603 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c0bf8e0 0x7f132c0c42d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- 172.21.15.118:0/3052908603 >> 172.21.15.118:0/3052908603 conn(0x7f132c06f4b0 msgr2=0x7f132c0c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- 172.21.15.118:0/3052908603 shutdown_connections 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.896+0000 7f13324dc700 1 -- 172.21.15.118:0/3052908603 wait complete. 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 Processor -- start 2024-08-18T13:53:06.902 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 -- start start 2024-08-18T13:53:06.904 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f132c0bf8e0 0x7f132c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f132c14c030 0x7f132c1487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c14c400 0x7f132c148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c067da0 con 0x7f132c14c400 2024-08-18T13:53:06.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c149c90 con 0x7f132c14c030 2024-08-18T13:53:06.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c159320 con 0x7f132c0bf8e0 2024-08-18T13:53:06.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c14c400 0x7f132c148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c14c400 0x7f132c148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41332/0 (socket says 172.21.15.118:41332) 2024-08-18T13:53:06.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 -- 172.21.15.118:0/1746600266 learned_addr learned my addr 172.21.15.118:0/1746600266 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:06.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1330cd9700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f132c14c030 0x7f132c1487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13314da700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f132c0bf8e0 0x7f132c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 -- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f132c0bf8e0 msgr2=0x7f132c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f132c0bf8e0 0x7f132c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 -- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f132c14c030 msgr2=0x7f132c1487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:06.906 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f132c14c030 0x7f132c1487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 -- 172.21.15.118:0/1746600266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324009ae0 con 0x7f132c14c400 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1330cd9700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f132c14c030 0x7f132c1487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13314da700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f132c0bf8e0 0x7f132c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f1331cdb700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c14c400 0x7f132c148d20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f132400b6c0 tx=0x7f1324009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f131d7fa700 1 -- 172.21.15.118:0/1746600266 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324008e90 con 0x7f132c14c400 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f131d7fa700 1 -- 172.21.15.118:0/1746600266 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f132401c410 con 0x7f132c14c400 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132c1595c0 con 0x7f132c14c400 2024-08-18T13:53:06.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f131d7fa700 1 -- 172.21.15.118:0/1746600266 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132401c7d0 con 0x7f132c14c400 2024-08-18T13:53:06.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.900+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132c067470 con 0x7f132c14c400 2024-08-18T13:53:06.908 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.904+0000 7f131d7fa700 1 -- 172.21.15.118:0/1746600266 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1324003040 con 0x7f132c14c400 2024-08-18T13:53:06.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.904+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132c067f50 con 0x7f132c14c400 2024-08-18T13:53:06.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.904+0000 7f131d7fa700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1310060860 0x7f1310062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:06.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.904+0000 7f131d7fa700 1 -- 172.21.15.118:0/1746600266 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7261+0+0 (secure 0 0 0) 0x7f132407ebe0 con 0x7f132c14c400 2024-08-18T13:53:06.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.904+0000 7f13314da700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1310060860 0x7f1310062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:06.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.904+0000 7f13314da700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1310060860 0x7f1310062d10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1320001820 tx=0x7f1320009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:06.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:06.912+0000 7f131d7fa700 1 -- 172.21.15.118:0/1746600266 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f132c067f50 con 0x7f132c14c400 2024-08-18T13:53:07.113 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.108+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f132c067f50 con 0x7f132c14c400 2024-08-18T13:53:07.776 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.772+0000 7f131d7fa700 1 -- 172.21.15.118:0/1746600266 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f132c067f50 con 0x7f132c14c400 2024-08-18T13:53:07.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1310060860 msgr2=0x7f1310062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:07.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1310060860 0x7f1310062d10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1320001820 tx=0x7f1320009040 comp rx=0 tx=0).stop 2024-08-18T13:53:07.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c14c400 msgr2=0x7f132c148d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:07.783 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c14c400 0x7f132c148d20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f132400b6c0 tx=0x7f1324009840 comp rx=0 tx=0).stop 2024-08-18T13:53:07.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 shutdown_connections 2024-08-18T13:53:07.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f1310060860 0x7f1310062d10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f132c14c400 0x7f132c148d20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f132c14c030 0x7f132c1487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 --2- 172.21.15.118:0/1746600266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f132c0bf8e0 0x7f132c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.784 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 >> 172.21.15.118:0/1746600266 conn(0x7f132c06f4b0 msgr2=0x7f132c0c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:07.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 shutdown_connections 2024-08-18T13:53:07.785 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.780+0000 7f13324dc700 1 -- 172.21.15.118:0/1746600266 wait complete. 2024-08-18T13:53:07.796 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-18T13:53:07.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.904+0000 7ff8bf59e700 1 Processor -- start 2024-08-18T13:53:07.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.904+0000 7ff8bf59e700 1 -- start start 2024-08-18T13:53:07.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff8c00c0b00 0x7ff8c0068820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:07.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0155c90 0x7ff8c00c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:07.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c00c15e0 con 0x7ff8c00c8d40 2024-08-18T13:53:07.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c00c7bf0 con 0x7ff8c00c0b00 2024-08-18T13:53:07.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01561b0 con 0x7ff8c0155c90 2024-08-18T13:53:07.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff8c00c0b00 0x7ff8c0068820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:07.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7ff8c00c0b00 0x7ff8c0068820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41334/0 (socket says 172.21.15.118:41334) 2024-08-18T13:53:07.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 -- 172.21.15.118:0/2727234690 learned_addr learned my addr 172.21.15.118:0/2727234690 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:07.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bdd9b700 1 --2- 172.21.15.118:0/2727234690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0155c90 0x7ff8c00c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:07.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 -- 172.21.15.118:0/2727234690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0155c90 msgr2=0x7ff8c00c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:07.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 --2- 172.21.15.118:0/2727234690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0155c90 0x7ff8c00c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 -- 172.21.15.118:0/2727234690 >> v1:172.21.15.178:6789/0 conn(0x7ff8c00c8d40 legacy=0x7ff8c00c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:07.912 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 -- 172.21.15.118:0/2727234690 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00c7000 con 0x7ff8c00c0b00 2024-08-18T13:53:07.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8be59c700 1 --2- 172.21.15.118:0/2727234690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c00c0b00 0x7ff8c0068820 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099b0 tx=0x7ff8b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b99dae315af479 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:07.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bd59a700 1 -- 172.21.15.118:0/2727234690 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4009ea0 con 0x7ff8c00c0b00 2024-08-18T13:53:07.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bd59a700 1 -- 172.21.15.118:0/2727234690 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff8b400b8f0 con 0x7ff8c00c0b00 2024-08-18T13:53:07.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bd59a700 1 -- 172.21.15.118:0/2727234690 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b400bcb0 con 0x7ff8c00c0b00 2024-08-18T13:53:07.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- 172.21.15.118:0/2727234690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c00c0b00 msgr2=0x7ff8c0068820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:07.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/2727234690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c00c0b00 0x7ff8c0068820 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099b0 tx=0x7ff8b4009250 comp rx=0 tx=0).stop 2024-08-18T13:53:07.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- 172.21.15.118:0/2727234690 shutdown_connections 2024-08-18T13:53:07.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/2727234690 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c0155c90 0x7ff8c00c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/2727234690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c00c0b00 0x7ff8c0068820 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- 172.21.15.118:0/2727234690 >> 172.21.15.118:0/2727234690 conn(0x7ff8c00cc630 msgr2=0x7ff8c00c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:07.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- 172.21.15.118:0/2727234690 shutdown_connections 2024-08-18T13:53:07.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.908+0000 7ff8bf59e700 1 -- 172.21.15.118:0/2727234690 wait complete. 2024-08-18T13:53:07.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 Processor -- start 2024-08-18T13:53:07.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 -- start start 2024-08-18T13:53:07.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c00c0b00 0x7ff8c00cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:07.917 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00c8d40 0x7ff8c0149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:07.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c0155c90 0x7ff8c0158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:07.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c014c280 con 0x7ff8c0155c90 2024-08-18T13:53:07.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c014d680 con 0x7ff8c00c0b00 2024-08-18T13:53:07.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c014d3c0 con 0x7ff8c00c8d40 2024-08-18T13:53:07.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c0155c90 0x7ff8c0158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:07.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c0155c90 0x7ff8c0158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41348/0 (socket says 172.21.15.118:41348) 2024-08-18T13:53:07.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 -- 172.21.15.118:0/750941346 learned_addr learned my addr 172.21.15.118:0/750941346 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bdd9b700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00c8d40 0x7ff8c0149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8be59c700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c00c0b00 0x7ff8c00cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 -- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00c8d40 msgr2=0x7ff8c0149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00c8d40 0x7ff8c0149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 -- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c00c0b00 msgr2=0x7ff8c00cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c00c0b00 0x7ff8c00cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 -- 172.21.15.118:0/750941346 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40094f0 con 0x7ff8c0155c90 2024-08-18T13:53:07.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8be59c700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c00c0b00 0x7ff8c00cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T13:53:07.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bed9d700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c0155c90 0x7ff8c0158c10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800c910 tx=0x7ff8b8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:07.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8ae7fc700 1 -- 172.21.15.118:0/750941346 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8012630 con 0x7ff8c0155c90 2024-08-18T13:53:07.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c00cb520 con 0x7ff8c0155c90 2024-08-18T13:53:07.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8ae7fc700 1 -- 172.21.15.118:0/750941346 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff8b8012790 con 0x7ff8c0155c90 2024-08-18T13:53:07.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8ae7fc700 1 -- 172.21.15.118:0/750941346 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8012b50 con 0x7ff8c0155c90 2024-08-18T13:53:07.920 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c014bf10 con 0x7ff8c0155c90 2024-08-18T13:53:07.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.912+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c0071410 con 0x7ff8c0155c90 2024-08-18T13:53:07.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.916+0000 7ff8ae7fc700 1 -- 172.21.15.118:0/750941346 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff8b801b400 con 0x7ff8c0155c90 2024-08-18T13:53:07.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.916+0000 7ff8ae7fc700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff8a0060860 0x7ff8a0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:07.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.916+0000 7ff8be59c700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff8a0060860 0x7ff8a0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:07.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.916+0000 7ff8ae7fc700 1 -- 172.21.15.118:0/750941346 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7261+0+0 (secure 0 0 0) 0x7ff8b807d530 con 0x7ff8c0155c90 2024-08-18T13:53:07.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.916+0000 7ff8be59c700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff8a0060860 0x7ff8a0062d10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099b0 tx=0x7ff8b4009250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:07.923 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:07.920+0000 7ff8ae7fc700 1 -- 172.21.15.118:0/750941346 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff8b8050e80 con 0x7ff8c0155c90 2024-08-18T13:53:08.087 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.084+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff8c00c15e0 con 0x7ff8c0155c90 2024-08-18T13:53:08.781 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.776+0000 7ff8ae7fc700 1 -- 172.21.15.118:0/750941346 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff8c00c15e0 con 0x7ff8c0155c90 2024-08-18T13:53:08.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff8a0060860 msgr2=0x7ff8a0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:08.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff8a0060860 0x7ff8a0062d10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40099b0 tx=0x7ff8b4009250 comp rx=0 tx=0).stop 2024-08-18T13:53:08.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c0155c90 msgr2=0x7ff8c0158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:08.786 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c0155c90 0x7ff8c0158c10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800c910 tx=0x7ff8b8001710 comp rx=0 tx=0).stop 2024-08-18T13:53:08.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 shutdown_connections 2024-08-18T13:53:08.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7ff8a0060860 0x7ff8a0062d10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c0155c90 0x7ff8c0158c10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8c00c8d40 0x7ff8c0149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 --2- 172.21.15.118:0/750941346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8c00c0b00 0x7ff8c00cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 >> 172.21.15.118:0/750941346 conn(0x7ff8c00cc630 msgr2=0x7ff8c01485e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:08.787 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 shutdown_connections 2024-08-18T13:53:08.788 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.784+0000 7ff8bf59e700 1 -- 172.21.15.118:0/750941346 wait complete. 2024-08-18T13:53:08.788 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_metadata' removed 2024-08-18T13:53:08.801 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-18T13:53:08.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c222bf700 1 Processor -- start 2024-08-18T13:53:08.905 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c222bf700 1 -- start start 2024-08-18T13:53:08.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c222bf700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9c1c0c0cf0 0x7f9c1c155d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:08.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c222bf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c14efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:08.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c222bf700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c0698b0 con 0x7f9c1c0c10c0 2024-08-18T13:53:08.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c222bf700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c156250 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.907 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c222bf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c06a450 con 0x7f9c1c0cc430 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c1bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9c1c0c0cf0 0x7f9c1c155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c2085c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c14efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c1bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9c1c0c0cf0 0x7f9c1c155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.118:56684/0 (socket says 172.21.15.118:56684) 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c2085c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c14efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.118:34052/0 (socket says 172.21.15.118:34052) 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c1bfff700 1 -- 172.21.15.118:0/3539855123 learned_addr learned my addr 172.21.15.118:0/3539855123 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c13fff700 1 --1- 172.21.15.118:0/3539855123 >> v1:172.21.15.178:6789/0 conn(0x7f9c1c0c10c0 0x7f9c1c069d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.118:38454/0 (socket says 172.21.15.118:38454) 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c2085c700 1 -- 172.21.15.118:0/3539855123 >> v2:172.21.15.178:3300/0 conn(0x7f9c1c0c0cf0 msgr2=0x7f9c1c155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:08.909 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c2085c700 1 --2- 172.21.15.118:0/3539855123 >> v2:172.21.15.178:3300/0 conn(0x7f9c1c0c0cf0 0x7f9c1c155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c2085c700 1 -- 172.21.15.118:0/3539855123 >> v1:172.21.15.178:6789/0 conn(0x7f9c1c0c10c0 legacy=0x7f9c1c069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:08.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c2085c700 1 -- 172.21.15.118:0/3539855123 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c0ccb70 con 0x7f9c1c0cc430 2024-08-18T13:53:08.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c1bfff700 1 --2- 172.21.15.118:0/3539855123 >> v2:172.21.15.178:3300/0 conn(0x7f9c1c0c0cf0 0x7f9c1c155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T13:53:08.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c2085c700 1 --2- 172.21.15.118:0/3539855123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c14efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009e60 tx=0x7f9c14009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb73801a2a09e55 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:08.910 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c1b7fe700 1 -- 172.21.15.118:0/3539855123 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400e070 con 0x7f9c1c0cc430 2024-08-18T13:53:08.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c1b7fe700 1 -- 172.21.15.118:0/3539855123 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9c1400bbf0 con 0x7f9c1c0cc430 2024-08-18T13:53:08.911 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.904+0000 7f9c1b7fe700 1 -- 172.21.15.118:0/3539855123 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14012430 con 0x7f9c1c0cc430 2024-08-18T13:53:08.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- 172.21.15.118:0/3539855123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 msgr2=0x7f9c1c14efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:08.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 --2- 172.21.15.118:0/3539855123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c14efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009e60 tx=0x7f9c14009780 comp rx=0 tx=0).stop 2024-08-18T13:53:08.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- 172.21.15.118:0/3539855123 shutdown_connections 2024-08-18T13:53:08.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 --2- 172.21.15.118:0/3539855123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c14efc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.913 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 --2- 172.21.15.118:0/3539855123 >> v2:172.21.15.178:3300/0 conn(0x7f9c1c0c0cf0 0x7f9c1c155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- 172.21.15.118:0/3539855123 >> 172.21.15.118:0/3539855123 conn(0x7f9c1c067df0 msgr2=0x7f9c1c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:08.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- 172.21.15.118:0/3539855123 shutdown_connections 2024-08-18T13:53:08.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- 172.21.15.118:0/3539855123 wait complete. 2024-08-18T13:53:08.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 Processor -- start 2024-08-18T13:53:08.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- start start 2024-08-18T13:53:08.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c1c0c0cf0 0x7f9c1c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:08.914 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0c10c0 0x7f9c1c0cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0ca900 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0b18a0 con 0x7f9c1c0c10c0 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c222bf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c0ca660 con 0x7f9c1c0cc430 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c1bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c1c0c0cf0 0x7f9c1c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c1bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c1c0c0cf0 0x7f9c1c0b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41364/0 (socket says 172.21.15.118:41364) 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c1bfff700 1 -- 172.21.15.118:0/261397655 learned_addr learned my addr 172.21.15.118:0/261397655 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:08.915 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c2085c700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:08.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.908+0000 7f9c13fff700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0c10c0 0x7f9c1c0cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:08.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c1bfff700 1 -- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 msgr2=0x7f9c1c144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:08.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c1bfff700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c1bfff700 1 -- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0c10c0 msgr2=0x7f9c1c0cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:08.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c1bfff700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0c10c0 0x7f9c1c0cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:08.916 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c1bfff700 1 -- 172.21.15.118:0/261397655 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0094f0 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c1bfff700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c1c0c0cf0 0x7f9c1c0b0ba0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00ec50 tx=0x7f9c0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:08.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c18ff9700 1 -- 172.21.15.118:0/261397655 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00c850 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c18ff9700 1 -- 172.21.15.118:0/261397655 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9c0c00c9b0 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c14009a20 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c18ff9700 1 -- 172.21.15.118:0/261397655 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00cd50 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c0c9e60 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.918 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1c0642f0 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c18ff9700 1 -- 172.21.15.118:0/261397655 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9c0c013680 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.912+0000 7f9c18ff9700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9bfc0608e0 0x7f9bfc062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:08.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.916+0000 7f9c13fff700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9bfc0608e0 0x7f9bfc062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:08.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.916+0000 7f9c18ff9700 1 -- 172.21.15.118:0/261397655 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6823+0+0 (secure 0 0 0) 0x7f9c0c07d460 con 0x7f9c1c0c0cf0 2024-08-18T13:53:08.919 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.916+0000 7f9c13fff700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9bfc0608e0 0x7f9bfc062d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0c25f0 tx=0x7f9c04007eb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:08.922 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:08.920+0000 7f9c18ff9700 1 -- 172.21.15.118:0/261397655 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9c0c04d430 con 0x7f9c1c0c0cf0 2024-08-18T13:53:09.088 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.084+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c1c0c78f0 con 0x7f9c1c0c0cf0 2024-08-18T13:53:09.801 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.796+0000 7f9c18ff9700 1 -- 172.21.15.118:0/261397655 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9c0c050a50 con 0x7f9c1c0c0cf0 2024-08-18T13:53:09.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9bfc0608e0 msgr2=0x7f9bfc062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:09.804 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9bfc0608e0 0x7f9bfc062d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0c25f0 tx=0x7f9c04007eb0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c1c0c0cf0 msgr2=0x7f9c1c0b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:09.805 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c1c0c0cf0 0x7f9c1c0b0ba0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00ec50 tx=0x7f9c0c009250 comp rx=0 tx=0).stop 2024-08-18T13:53:09.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 shutdown_connections 2024-08-18T13:53:09.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7f9bfc0608e0 0x7f9bfc062d90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c1c0cc430 0x7f9c1c144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c1c0c10c0 0x7f9c1c0cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 --2- 172.21.15.118:0/261397655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9c1c0c0cf0 0x7f9c1c0b0ba0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.800+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 >> 172.21.15.118:0/261397655 conn(0x7f9c1c067df0 msgr2=0x7f9c1c0c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:09.806 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.804+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 shutdown_connections 2024-08-18T13:53:09.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.804+0000 7f9c222bf700 1 -- 172.21.15.118:0/261397655 wait complete. 2024-08-18T13:53:09.807 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_data' removed 2024-08-18T13:53:09.819 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-08-18T13:53:09.932 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.928+0000 7feb5880a700 1 Processor -- start 2024-08-18T13:53:09.932 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.928+0000 7feb5880a700 1 -- start start 2024-08-18T13:53:09.934 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7feb54068820 0x7feb540c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:09.934 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb540c8d40 0x7feb5414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:09.934 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb540c7980 con 0x7feb540c0b00 2024-08-18T13:53:09.934 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- --> v2:172.21.15.118:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb540c2490 con 0x7feb54068820 2024-08-18T13:53:09.934 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb54156280 con 0x7feb540c8d40 2024-08-18T13:53:09.935 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7feb54068820 0x7feb540c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:09.935 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 --2- >> v2:172.21.15.118:3300/0 conn(0x7feb54068820 0x7feb540c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41370/0 (socket says 172.21.15.118:41370) 2024-08-18T13:53:09.935 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 -- 172.21.15.118:0/594199468 learned_addr learned my addr 172.21.15.118:0/594199468 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:09.935 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb53fff700 1 --2- 172.21.15.118:0/594199468 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb540c8d40 0x7feb5414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 -- 172.21.15.118:0/594199468 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb540c8d40 msgr2=0x7feb5414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 --2- 172.21.15.118:0/594199468 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb540c8d40 0x7feb5414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 -- 172.21.15.118:0/594199468 >> v1:172.21.15.178:6789/0 conn(0x7feb540c0b00 legacy=0x7feb540c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 -- 172.21.15.118:0/594199468 --> v2:172.21.15.118:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb54155ce0 con 0x7feb54068820 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb52ffd700 1 --2- 172.21.15.118:0/594199468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb54068820 0x7feb540c0ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7feb440099b0 tx=0x7feb44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeefd910cd6b328c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb527fc700 1 -- 172.21.15.118:0/594199468 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44009ea0 con 0x7feb54068820 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb527fc700 1 -- 172.21.15.118:0/594199468 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7feb4400b860 con 0x7feb54068820 2024-08-18T13:53:09.937 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb527fc700 1 -- 172.21.15.118:0/594199468 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400bc20 con 0x7feb54068820 2024-08-18T13:53:09.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- 172.21.15.118:0/594199468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb54068820 msgr2=0x7feb540c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:09.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 --2- 172.21.15.118:0/594199468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb54068820 0x7feb540c0ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7feb440099b0 tx=0x7feb44009250 comp rx=0 tx=0).stop 2024-08-18T13:53:09.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- 172.21.15.118:0/594199468 shutdown_connections 2024-08-18T13:53:09.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 --2- 172.21.15.118:0/594199468 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb540c8d40 0x7feb5414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 --2- 172.21.15.118:0/594199468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb54068820 0x7feb540c0ed0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- 172.21.15.118:0/594199468 >> 172.21.15.118:0/594199468 conn(0x7feb540cc630 msgr2=0x7feb540c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:09.938 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- 172.21.15.118:0/594199468 shutdown_connections 2024-08-18T13:53:09.939 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.932+0000 7feb5880a700 1 -- 172.21.15.118:0/594199468 wait complete. 2024-08-18T13:53:09.943 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 Processor -- start 2024-08-18T13:53:09.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 -- start start 2024-08-18T13:53:09.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb54068820 0x7feb541692d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:09.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb540c0b00 0x7feb541547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:09.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb540c8d40 0x7feb540af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:09.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540b0f00 con 0x7feb540c0b00 2024-08-18T13:53:09.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540af0f0 con 0x7feb540c8d40 2024-08-18T13:53:09.944 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb5880a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb54165490 con 0x7feb54068820 2024-08-18T13:53:09.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb52ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb540c0b00 0x7feb541547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:09.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb52ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb540c0b00 0x7feb541547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:41376/0 (socket says 172.21.15.118:41376) 2024-08-18T13:53:09.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb52ffd700 1 -- 172.21.15.118:0/944546932 learned_addr learned my addr 172.21.15.118:0/944546932 (peer_addr_for_me v2:172.21.15.118:0/0) 2024-08-18T13:53:09.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.936+0000 7feb537fe700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb54068820 0x7feb541692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:09.945 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb53fff700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb540c8d40 0x7feb540af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb52ffd700 1 -- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb54068820 msgr2=0x7feb541692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb52ffd700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb54068820 0x7feb541692d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb52ffd700 1 -- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb540c8d40 msgr2=0x7feb540af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb52ffd700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb540c8d40 0x7feb540af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb52ffd700 1 -- 172.21.15.118:0/944546932 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440094f0 con 0x7feb540c0b00 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb52ffd700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb540c0b00 0x7feb541547c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7feb440090b0 tx=0x7feb44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb4b7fe700 1 -- 172.21.15.118:0/944546932 <== mon.0 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400bbe0 con 0x7feb540c0b00 2024-08-18T13:53:09.946 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb4b7fe700 1 -- 172.21.15.118:0/944546932 <== mon.0 v2:172.21.15.118:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7feb4401f070 con 0x7feb540c0b00 2024-08-18T13:53:09.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb54165fd0 con 0x7feb540c0b00 2024-08-18T13:53:09.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb4b7fe700 1 -- 172.21.15.118:0/944546932 <== mon.0 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb440266f0 con 0x7feb540c0b00 2024-08-18T13:53:09.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb541656a0 con 0x7feb540c0b00 2024-08-18T13:53:09.947 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb54071410 con 0x7feb540c0b00 2024-08-18T13:53:09.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb4b7fe700 1 -- 172.21.15.118:0/944546932 <== mon.0 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7feb44026a60 con 0x7feb540c0b00 2024-08-18T13:53:09.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb4b7fe700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feb34060860 0x7feb34062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T13:53:09.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb4b7fe700 1 -- 172.21.15.118:0/944546932 <== mon.0 v2:172.21.15.118:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6293+0+0 (secure 0 0 0) 0x7feb44021070 con 0x7feb540c0b00 2024-08-18T13:53:09.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.940+0000 7feb537fe700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feb34060860 0x7feb34062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T13:53:09.948 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.944+0000 7feb537fe700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feb34060860 0x7feb34062d10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0037a0 tx=0x7feb3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T13:53:09.950 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:09.948+0000 7feb4b7fe700 1 -- 172.21.15.118:0/944546932 <== mon.0 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7feb44080e60 con 0x7feb540c0b00 2024-08-18T13:53:10.132 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.128+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb5406a4d0 con 0x7feb540c0b00 2024-08-18T13:53:10.802 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.800+0000 7feb4b7fe700 1 -- 172.21.15.118:0/944546932 <== mon.0 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v58) v1 ==== 178+0+0 (secure 0 0 0) 0x7feb4402b0d0 con 0x7feb540c0b00 2024-08-18T13:53:10.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.804+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feb34060860 msgr2=0x7feb34062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:10.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.804+0000 7feb5880a700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feb34060860 0x7feb34062d10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0037a0 tx=0x7feb3c009040 comp rx=0 tx=0).stop 2024-08-18T13:53:10.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.804+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb540c0b00 msgr2=0x7feb541547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T13:53:10.807 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.804+0000 7feb5880a700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb540c0b00 0x7feb541547c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7feb440090b0 tx=0x7feb44009250 comp rx=0 tx=0).stop 2024-08-18T13:53:10.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 shutdown_connections 2024-08-18T13:53:10.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:6832/24192,v1:172.21.15.118:6833/24192] conn(0x7feb34060860 0x7feb34062d10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:10.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb540c8d40 0x7feb540af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:10.809 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7feb540c0b00 0x7feb541547c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:10.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 --2- 172.21.15.118:0/944546932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb54068820 0x7feb541692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T13:53:10.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 >> 172.21.15.118:0/944546932 conn(0x7feb540cc630 msgr2=0x7feb540cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T13:53:10.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 shutdown_connections 2024-08-18T13:53:10.810 INFO:teuthology.orchestra.run.smithi118.stderr:2024-08-18T13:53:10.808+0000 7feb5880a700 1 -- 172.21.15.118:0/944546932 wait complete. 2024-08-18T13:53:10.811 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_data_ec' removed 2024-08-18T13:53:10.826 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-18T13:53:10.827 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-18T13:53:10.827 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:10.837 INFO:tasks.ceph.mds.a:Stopped 2024-08-18T13:53:10.838 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-18T13:53:10.838 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:10.849 INFO:tasks.ceph.mds.c:Stopped 2024-08-18T13:53:10.850 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-08-18T13:53:10.850 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:10.860 INFO:tasks.ceph.mds.e:Stopped 2024-08-18T13:53:10.860 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-18T13:53:10.861 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:10.871 INFO:tasks.ceph.mds.b:Stopped 2024-08-18T13:53:10.872 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-18T13:53:10.872 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:10.924 INFO:tasks.ceph.mds.d:Stopped 2024-08-18T13:53:10.925 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-08-18T13:53:10.925 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:10.935 INFO:tasks.ceph.mds.f:Stopped 2024-08-18T13:53:10.935 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-18T13:53:10.936 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-18T13:53:10.936 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.009 INFO:tasks.ceph.osd.0:Stopped 2024-08-18T13:53:11.010 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-18T13:53:11.010 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.097 INFO:tasks.ceph.osd.1:Stopped 2024-08-18T13:53:11.098 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-18T13:53:11.098 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.199 INFO:tasks.ceph.osd.2:Stopped 2024-08-18T13:53:11.200 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-18T13:53:11.200 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.299 INFO:tasks.ceph.osd.3:Stopped 2024-08-18T13:53:11.299 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-18T13:53:11.300 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.366 INFO:tasks.ceph.osd.4:Stopped 2024-08-18T13:53:11.369 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-18T13:53:11.369 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.449 INFO:tasks.ceph.osd.5:Stopped 2024-08-18T13:53:11.450 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-08-18T13:53:11.450 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.519 INFO:tasks.ceph.osd.6:Stopped 2024-08-18T13:53:11.519 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-08-18T13:53:11.519 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.603 INFO:tasks.ceph.osd.7:Stopped 2024-08-18T13:53:11.603 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-18T13:53:11.604 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-08-18T13:53:11.604 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.648 INFO:tasks.ceph.mgr.y:Stopped 2024-08-18T13:53:11.648 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-18T13:53:11.649 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.673 INFO:tasks.ceph.mgr.x:Stopped 2024-08-18T13:53:11.673 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-18T13:53:11.673 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-18T13:53:11.674 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.693 INFO:tasks.ceph.mon.a:Stopped 2024-08-18T13:53:11.693 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-18T13:53:11.693 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.711 INFO:tasks.ceph.mon.b:Stopped 2024-08-18T13:53:11.711 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-18T13:53:11.711 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T13:53:11.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-08-18T13:53:11.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-08-18T13:53:11.773 INFO:tasks.ceph.mon.c:Stopped 2024-08-18T13:53:11.773 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-18T13:53:11.774 DEBUG:teuthology.orchestra.run.smithi118:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'Replacing daemon mds' | head -n 1 2024-08-18T13:53:11.801 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:53:11.801 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-18T13:53:12.442 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:53:12.442 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-18T13:53:12.710 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:53:12.710 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-18T13:53:12.943 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:53:12.944 DEBUG:teuthology.orchestra.run.smithi118:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-18T13:53:13.159 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:53:13.159 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-18T13:53:13.915 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:53:13.915 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-18T13:53:14.196 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:53:14.196 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-08-18T13:53:14.411 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:53:14.411 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-08-18T13:53:14.596 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:53:14.598 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:53:14.610 INFO:tasks.ceph:Archiving mon data... 2024-08-18T13:53:14.612 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/data/mon.a.tgz 2024-08-18T13:53:14.612 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-08-18T13:53:14.619 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.SLUQbJeKtx 2024-08-18T13:53:14.619 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SLUQbJeKtx 2024-08-18T13:53:14.784 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.SLUQbJeKtx 2024-08-18T13:53:14.806 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.SLUQbJeKtx is 330KB 2024-08-18T13:53:14.867 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.SLUQbJeKtx 2024-08-18T13:53:14.874 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/data/mon.b.tgz 2024-08-18T13:53:14.874 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-08-18T13:53:14.881 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.IovuotfJSi 2024-08-18T13:53:14.881 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.IovuotfJSi 2024-08-18T13:53:14.971 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.IovuotfJSi 2024-08-18T13:53:14.997 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.IovuotfJSi is 336KB 2024-08-18T13:53:15.057 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.IovuotfJSi 2024-08-18T13:53:15.064 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/data/mon.c.tgz 2024-08-18T13:53:15.064 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-08-18T13:53:15.070 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.yMQ4WQWDtW 2024-08-18T13:53:15.071 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.yMQ4WQWDtW 2024-08-18T13:53:15.154 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.yMQ4WQWDtW 2024-08-18T13:53:15.217 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.yMQ4WQWDtW is 336KB 2024-08-18T13:53:15.237 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.yMQ4WQWDtW 2024-08-18T13:53:15.242 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-18T13:53:15.243 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-18T13:53:15.245 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-18T13:53:15.262 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-08-18T13:53:15.262 DEBUG:teuthology.orchestra.run.smithi118:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-18T13:53:15.265 DEBUG:teuthology.orchestra.run.smithi178:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-18T13:53:15.291 INFO:teuthology.orchestra.run.smithi118.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-18T13:53:15.340 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-18T13:53:15.347 INFO:tasks.ceph:Archiving crash dumps... 2024-08-18T13:53:15.349 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/remote/smithi118/crash 2024-08-18T13:53:15.350 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-18T13:53:15.366 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/remote/smithi178/crash 2024-08-18T13:53:15.367 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-18T13:53:15.384 INFO:tasks.ceph:Shutting down logrotate 2024-08-18T13:53:15.384 DEBUG:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:53:15.387 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-18T13:53:15.485 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-18T13:53:15.501 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-18T13:53:15.517 INFO:tasks.ceph:Compressing logs... 2024-08-18T13:53:15.517 DEBUG:teuthology.orchestra.run.smithi118:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-18T13:53:15.547 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-18T13:53:15.561 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38748.log 2024-08-18T13:53:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38477.log 2024-08-18T13:53:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38748.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38748.log.gz 2024-08-18T13:53:15.562 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34142.log 2024-08-18T13:53:15.563 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31303.log 2024-08-18T13:53:15.563 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30659.log 2024-08-18T13:53:15.566 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.34142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38042.log 2024-08-18T13:53:15.566 INFO:teuthology.orchestra.run.smithi118.stderr: 89.3%/var/log/ceph/ceph-client.admin.31303.log: -- replaced with /var/log/ceph/ceph-client.admin.38477.log.gz 2024-08-18T13:53:15.566 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29864.log 2024-08-18T13:53:15.566 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34142.log.gz 2024-08-18T13:53:15.567 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31303.log.gz 2024-08-18T13:53:15.567 INFO:teuthology.orchestra.run.smithi118.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30659.log.gz 2024-08-18T13:53:15.567 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32019.log 2024-08-18T13:53:15.567 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38042.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38042.log.gz 2024-08-18T13:53:15.567 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37988.log 2024-08-18T13:53:15.567 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38640.log 2024-08-18T13:53:15.570 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37988.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29864.log.gz 2024-08-18T13:53:15.570 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32019.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.37988.log.gz-5 2024-08-18T13:53:15.570 INFO:teuthology.orchestra.run.smithi118.stderr:--verbose -- /var/log/ceph/ceph-client.admin.33924.log 2024-08-18T13:53:15.571 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32288.log 2024-08-18T13:53:15.571 INFO:teuthology.orchestra.run.smithi118.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32423.log -- replaced with /var/log/ceph/ceph-client.admin.32019.log.gz 2024-08-18T13:53:15.571 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-08-18T13:53:15.571 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38640.log: /var/log/ceph/ceph-client.admin.33924.log: /var/log/ceph/ceph-client.admin.32288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36109.log 2024-08-18T13:53:15.571 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32288.log.gz 2024-08-18T13:53:15.571 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38640.log.gz/var/log/ceph/ceph-client.admin.37071.log 2024-08-18T13:53:15.571 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36109.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.19295.log 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32423.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33924.log.gz 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37071.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36109.log.gz 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph.tmp-client.admin.19295.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.19295.log.gz 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37071.log.gz 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37549.log 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32423.log.gz 2024-08-18T13:53:15.572 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36574.log 2024-08-18T13:53:15.573 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.37549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37959.log 2024-08-18T13:53:15.573 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36574.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37549.log.gz 2024-08-18T13:53:15.576 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36574.log.gz 2024-08-18T13:53:15.576 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32666.log 2024-08-18T13:53:15.576 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29686.log 2024-08-18T13:53:15.576 INFO:teuthology.orchestra.run.smithi118.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37959.log.gz 2024-08-18T13:53:15.576 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32666.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32666.log.gz 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31092.log 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36136.log 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29686.log: /var/log/ceph/ceph-client.admin.31092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30875.log 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29686.log.gz 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31092.log.gz 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36136.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36136.log.gz 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31296.log 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31876.log 2024-08-18T13:53:15.577 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30875.log: /var/log/ceph/ceph-client.admin.31296.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30875.log.gz 2024-08-18T13:53:15.578 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39344.log 2024-08-18T13:53:15.578 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31768.log 2024-08-18T13:53:15.578 INFO:teuthology.orchestra.run.smithi118.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31296.log.gz 2024-08-18T13:53:15.578 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31876.log.gz 2024-08-18T13:53:15.578 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39344.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39344.log.gz 2024-08-18T13:53:15.578 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37354.log 2024-08-18T13:53:15.581 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24376.log 2024-08-18T13:53:15.581 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-08-18T13:53:15.581 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24243.log 2024-08-18T13:53:15.582 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24376.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-08-18T13:53:15.582 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.f.log: 88.1%/var/log/ceph/ceph-client.admin.24243.log: -- replaced with /var/log/ceph/ceph-client.admin.24376.log.gz 2024-08-18T13:53:15.582 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-18T13:53:15.582 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.d.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.24243.log.gz 2024-08-18T13:53:15.582 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24216.log 2024-08-18T13:53:15.587 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.31768.log: -- /var/log/ceph/ceph-client.admin.32693.log 2024-08-18T13:53:15.587 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31714.log 2024-08-18T13:53:15.587 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31768.log.gz 2024-08-18T13:53:15.587 INFO:teuthology.orchestra.run.smithi118.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37354.log.gz 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32693.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32693.log.gz 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34196.log 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31714.log.gz 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mds.c.log: 94.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38096.log 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.34196.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31552.log 2024-08-18T13:53:15.588 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36082.log 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34196.log.gz 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38096.log.gz 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30194.log 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31552.log: /var/log/ceph/ceph-client.admin.36082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31280.log 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31552.log.gz 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36082.log.gz 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.30194.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31930.log 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37017.log 2024-08-18T13:53:15.589 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30194.log.gz 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31280.log: /var/log/ceph/ceph-client.admin.31930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38883.log 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31280.log.gz 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31930.log.gz 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37017.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37017.log.gz 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37684.log 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34115.log 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38883.log: /var/log/ceph/ceph-client.admin.37684.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-08-18T13:53:15.590 INFO:teuthology.orchestra.run.smithi118.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38883.log.gz 2024-08-18T13:53:15.591 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37684.log.gz 2024-08-18T13:53:15.591 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.34115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33811.log 2024-08-18T13:53:15.591 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39209.log 2024-08-18T13:53:15.591 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6%/var/log/ceph/ceph-mds.a.log: -- replaced with /var/log/ceph/ceph-client.admin.34115.log.gz 2024-08-18T13:53:15.591 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32315.log 2024-08-18T13:53:15.591 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33811.log.gz 2024-08-18T13:53:15.591 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39209.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39209.log.gz 2024-08-18T13:53:15.597 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24295.log 2024-08-18T13:53:15.597 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-18T13:53:15.597 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24216.log: /var/log/ceph/ceph-client.admin.24295.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.24216.log.gz 2024-08-18T13:53:15.597 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24268.log 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.24295.log.gz 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24404.log 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.c.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.24268.log.gz 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24404.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24404.log.gz 2024-08-18T13:53:15.598 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-08-18T13:53:15.599 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.audit.log: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-18T13:53:15.599 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24349.log 2024-08-18T13:53:15.607 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24322.log 2024-08-18T13:53:15.607 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24349.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.24349.log.gz 2024-08-18T13:53:15.607 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-08-18T13:53:15.608 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24322.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-18T13:53:15.608 INFO:teuthology.orchestra.run.smithi178.stderr: 88.3%/var/log/ceph/ceph-mds.b.log: -- replaced with /var/log/ceph/ceph-client.admin.24322.log.gz 2024-08-18T13:53:15.608 INFO:teuthology.orchestra.run.smithi178.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-18T13:53:15.608 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-18T13:53:15.608 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.log: 94.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-08-18T13:53:15.610 INFO:teuthology.orchestra.run.smithi178.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-18T13:53:15.610 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-08-18T13:53:15.627 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36354.log 2024-08-18T13:53:15.627 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36601.log 2024-08-18T13:53:15.627 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32315.log: /var/log/ceph/ceph-client.admin.36354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29808.log 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32315.log.gz/var/log/ceph/ceph-client.admin.36601.log: 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36354.log.gz 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31687.log 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36601.log.gz 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37219.log 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29808.log: /var/log/ceph/ceph-client.admin.31687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37044.log 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-08-18T13:53:15.628 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31687.log.gz 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29808.log.gz 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30250.log 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31300.log 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37219.log.gz 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37044.log: /var/log/ceph/ceph-client.admin.30250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37711.log 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37044.log.gz 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30250.log.gz 2024-08-18T13:53:15.629 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24153.log 2024-08-18T13:53:15.630 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37100.log 2024-08-18T13:53:15.630 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31300.log.gz 2024-08-18T13:53:15.630 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37711.log: /var/log/ceph/ceph-client.admin.24153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32504.log 2024-08-18T13:53:15.630 INFO:teuthology.orchestra.run.smithi118.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.24153.log.gz 2024-08-18T13:53:15.630 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.37711.log.gz 2024-08-18T13:53:15.630 INFO:teuthology.orchestra.run.smithi118.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.39047.log 2024-08-18T13:53:15.630 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38585.log 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36936.log 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37100.log.gz 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39047.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32504.log.gz 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38531.log 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38585.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39047.log.gz 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36936.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38585.log.gz 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose 88.5%-- -- replaced with /var/log/ceph/ceph-client.admin.36936.log.gz/var/log/ceph/ceph-client.admin.35944.log 2024-08-18T13:53:15.631 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38531.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38126.log 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35944.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38531.log.gz 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30873.log 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35944.log.gz 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30866.log 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38126.log: /var/log/ceph/ceph-client.admin.30873.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38126.log.gz 2024-08-18T13:53:15.632 INFO:teuthology.orchestra.run.smithi118.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30873.log.gz 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35532.log 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37822.log 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37795.log 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30866.log.gz 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31308.log 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37822.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35532.log.gz 2024-08-18T13:53:15.633 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33897.log 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37822.log.gz 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30113.log 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37795.log.gz 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33757.log 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31308.log.gz 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33897.log.gz 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39236.log 2024-08-18T13:53:15.634 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30113.log.gz 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33757.log: gzip /var/log/ceph/ceph-client.admin.39236.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33676.log 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33757.log.gz 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37903.log 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39236.log.gz 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr: 89.2%/var/log/ceph/ceph-client.admin.33676.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz-5 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr:--verbose -- /var/log/ceph/ceph-client.admin.35753.log 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33676.log.gz 2024-08-18T13:53:15.635 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30551.log 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31091.log 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37903.log.gz 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30551.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35753.log.gz 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30466.log 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30551.log.gz 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30221.log 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36963.log 2024-08-18T13:53:15.636 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30466.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31091.log.gz 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38207.log 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30221.log.gz 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31085.log 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30869.log 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38207.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36963.log.gz 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31085.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38207.log.gz 2024-08-18T13:53:15.637 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32531.log /var/log/ceph/ceph-client.admin.30869.log: 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.30466.log.gz 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31085.log.gz 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30870.log 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35863.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30869.log.gz 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33870.log 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gz 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30870.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32531.log.gz 2024-08-18T13:53:15.638 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30084.log 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33870.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30870.log.gz 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33784.log 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33870.log.gz 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31849.log 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30084.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33784.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30084.log.gz 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31525.log 2024-08-18T13:53:15.639 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33784.log.gz 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31849.log.gz 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34169.log 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31525.log.gz 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37630.log 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.34169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30632.log 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38015.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34169.log.gz 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36853.log 2024-08-18T13:53:15.640 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36633.log 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37630.log.gz 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30632.log.gz 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38344.log 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31795.log 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36633.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36853.log.gz 2024-08-18T13:53:15.641 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35917.log 2024-08-18T13:53:15.642 INFO:teuthology.orchestra.run.smithi118.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36633.log.gz 2024-08-18T13:53:15.642 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30001.log 2024-08-18T13:53:15.642 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38344.log.gz 2024-08-18T13:53:15.642 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31795.log.gz 2024-08-18T13:53:15.642 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30860.log 2024-08-18T13:53:15.642 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30057.log 2024-08-18T13:53:15.642 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30001.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35917.log.gz 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38504.log 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30001.log.gz 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30860.log.gz 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36716.log 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32234.log 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30057.log.gz 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-08-18T13:53:15.643 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36716.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38504.log.gz 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30140.log 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32234.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36716.log.gz 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38856.log 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32234.log.gz 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36163.log 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30140.log: /var/log/ceph/ceph-client.admin.38856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36909.log 2024-08-18T13:53:15.644 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30140.log.gz 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30767.log 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36163.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38856.log.gz 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38775.log 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36909.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36163.log.gz 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36909.log.gz 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31087.log 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38775.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30767.log.gz 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29945.log 2024-08-18T13:53:15.645 INFO:teuthology.orchestra.run.smithi118.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38775.log.gz 2024-08-18T13:53:15.646 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32369.log 2024-08-18T13:53:15.646 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-08-18T13:53:15.646 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31087.log.gz 2024-08-18T13:53:15.646 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29945.log: /var/log/ceph/ceph-client.admin.32369.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29945.log.gz 2024-08-18T13:53:15.646 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32369.log.gz 2024-08-18T13:53:15.646 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32477.log 2024-08-18T13:53:15.646 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37603.log 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38290.log 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32477.log: /var/log/ceph/ceph-client.admin.37603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30277.log 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32477.log.gz 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37603.log.gz 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39290.log 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38290.log: /var/log/ceph/ceph-client.admin.30277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30864.log 2024-08-18T13:53:15.647 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38290.log.gz 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30605.log 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39290.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30277.log.gz 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32639.log 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30864.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39290.log.gz 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37437.log 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30605.log.gz 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose 89.0%-- /var/log/ceph/ceph-client.admin.34034.log 2024-08-18T13:53:15.648 INFO:teuthology.orchestra.run.smithi118.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30864.log.gz 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32639.log: /var/log/ceph/ceph-client.admin.37437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32450.log 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.34034.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32639.log.gz 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37437.log.gz 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38317.log 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.34034.log.gz-5 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr:--verbose -- /var/log/ceph/ceph-client.admin.31089.log 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38937.log 2024-08-18T13:53:15.649 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38317.log: /var/log/ceph/ceph-client.admin.31089.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38317.log.gz 2024-08-18T13:53:15.650 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30821.log 2024-08-18T13:53:15.650 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32450.log.gz 2024-08-18T13:53:15.650 INFO:teuthology.orchestra.run.smithi118.stderr:gzip 88.5%-5 -- replaced with /var/log/ceph/ceph-client.admin.31089.log.gz--verbose 2024-08-18T13:53:15.650 INFO:teuthology.orchestra.run.smithi118.stderr:-- /var/log/ceph/ceph-client.admin.37576.log 2024-08-18T13:53:15.650 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31957.log 2024-08-18T13:53:15.650 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30821.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38937.log.gz 2024-08-18T13:53:15.650 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31579.log 2024-08-18T13:53:15.651 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37576.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30821.log.gz 2024-08-18T13:53:15.651 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-18T13:53:15.651 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31957.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37576.log.gz 2024-08-18T13:53:15.651 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38964.log 2024-08-18T13:53:15.651 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37493.log 2024-08-18T13:53:15.677 INFO:teuthology.orchestra.run.smithi118.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.31957.log.gz 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29723.log 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8%/var/log/ceph/ceph-client.admin.38964.log: -- replaced with /var/log/ceph/ceph-client.admin.31579.log.gz 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30713.log 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37493.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38964.log.gz 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35668.log 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37493.log.gz 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29723.log.gz 2024-08-18T13:53:15.678 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31822.log 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36408.log 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35668.log: gzip /var/log/ceph/ceph-client.admin.31822.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32612.log 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30713.log.gz 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35668.log.gz 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36408.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31822.log.gz 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37657.log 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36408.log.gz 2024-08-18T13:53:15.679 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32128.log 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35807.log 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37657.log: /var/log/ceph/ceph-client.0.32128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36217.log 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37657.log.gz 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32612.log.gz 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36660.log 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31301.log 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36217.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35807.log.gz 2024-08-18T13:53:15.680 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36381.log 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36660.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36217.log.gz 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31301.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36660.log.gz 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31301.log.gz 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39018.log 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36244.log 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36880.log 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39018.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36381.log.gz 2024-08-18T13:53:15.681 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36493.log 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39018.log.gz 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36190.log 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36244.log.gz 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35613.log 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36880.log.gz 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36190.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36493.log.gz 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36190.log.gz 2024-08-18T13:53:15.682 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37768.log 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35780.log 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29891.log 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37768.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35613.log.gz 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.35780.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38829.log 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37768.log.gz 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31090.log 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35780.log.gz 2024-08-18T13:53:15.683 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36011.log 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38991.log 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29891.log.gz 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31090.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38829.log.gz 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36011.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31090.log.gz 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35890.log 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36011.log.gz 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38261.log 2024-08-18T13:53:15.684 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38991.log: /var/log/ceph/ceph-client.admin.35890.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38991.log.gz 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35890.log.gz 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33703.log 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30876.log 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35836.log 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33703.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38261.log.gz 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30876.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33703.log.gz 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30876.log.gz 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36327.log 2024-08-18T13:53:15.685 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36743.log 2024-08-18T13:53:15.686 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35836.log.gz 2024-08-18T13:53:15.686 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36520.log 2024-08-18T13:53:15.686 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34007.log 2024-08-18T13:53:15.686 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36327.log.gz 2024-08-18T13:53:15.686 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36743.log: /var/log/ceph/ceph-client.admin.36520.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36743.log.gz 2024-08-18T13:53:15.686 INFO:teuthology.orchestra.run.smithi118.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36520.log.gz 2024-08-18T13:53:15.686 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36687.log 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.34007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29618.log 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38069.log 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34007.log.gz 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36687.log.gz 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31660.log 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36439.log 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7%/var/log/ceph/ceph-client.admin.38069.log: -- replaced with /var/log/ceph/ceph-client.admin.29618.log.gz 2024-08-18T13:53:15.687 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37327.log 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38069.log.gz 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31660.log.gz 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33840.log 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37410.log 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37327.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36439.log.gz 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36547.log 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr: 89.5% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33840.log.gz -- replaced with /var/log/ceph/ceph-client.admin.37327.log.gz 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-08-18T13:53:15.688 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37410.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37410.log.gz 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36990.log 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36298.log 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36547.log: /var/log/ceph/ceph-client.admin.36990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30578.log 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36547.log.gz 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36990.log.gz 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37849.log 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24278.log 2024-08-18T13:53:15.689 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36298.log.gz 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30578.log: /var/log/ceph/ceph-client.admin.37849.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30578.log.gz 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37849.log.gz 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29918.log 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31741.log 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.24278.log: /var/log/ceph/ceph-client.admin.29918.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.24278.log.gz 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30794.log 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29918.log.gz 2024-08-18T13:53:15.690 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31741.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31741.log.gz 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31088.log 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31903.log 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34061.log 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30794.log.gz 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32396.log 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31903.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31088.log.gz 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.34061.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31903.log.gz 2024-08-18T13:53:15.691 INFO:teuthology.orchestra.run.smithi118.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34061.log.gz 2024-08-18T13:53:15.692 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38398.log 2024-08-18T13:53:15.692 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35641.log 2024-08-18T13:53:15.692 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29972.log 2024-08-18T13:53:15.692 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32396.log.gz 2024-08-18T13:53:15.692 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38180.log 2024-08-18T13:53:15.692 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35641.log: /var/log/ceph/ceph-client.admin.29972.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38398.log.gz 2024-08-18T13:53:15.692 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35641.log.gz 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29972.log.gz 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31306.log 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35984.log 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38180.log: gzip /var/log/ceph/ceph-client.admin.31306.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32342.log 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38180.log.gz 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35984.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31306.log.gz 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38447.log 2024-08-18T13:53:15.693 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35984.log.gz 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32342.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32342.log.gz 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30524.log 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38447.log: /var/log/ceph/ceph-client.admin.35697.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38447.log.gz 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39317.log 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-08-18T13:53:15.694 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30524.log: /var/log/ceph/ceph-client.admin.39317.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30524.log.gz 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39317.log.gz 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36799.log 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36826.log: /var/log/ceph/ceph-client.admin.36799.log: 89.2% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36799.log.gz 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38721.log 2024-08-18T13:53:15.695 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38612.log 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37136.log 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38721.log.gz 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38612.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38612.log.gz 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37738.log 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.37136.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30686.log 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37738.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37136.log.gz 2024-08-18T13:53:15.696 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39074.log 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37738.log.gz 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30686.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30686.log.gz 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39103.log 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32720.log 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39074.log: gzip /var/log/ceph/ceph-client.admin.39103.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39128.log 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39074.log.gz 2024-08-18T13:53:15.697 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.32720.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34088.log 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39103.log.gz 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30030.log 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32720.log.gz 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32261.log 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.34088.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39128.log.gz/var/log/ceph/ceph-client.admin.30030.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34088.log.gz 2024-08-18T13:53:15.698 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30030.log.gz 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38153.log 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32261.log.gz 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38234.log 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38558.log 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38153.log: gzip /var/log/ceph/ceph-client.admin.38234.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31305.log 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38153.log.gz 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38234.log.gz 2024-08-18T13:53:15.699 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38558.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38558.log.gz 2024-08-18T13:53:15.700 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36045.log 2024-08-18T13:53:15.700 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31606.log 2024-08-18T13:53:15.700 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31305.log: /var/log/ceph/ceph-client.admin.36045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37932.log 2024-08-18T13:53:15.700 INFO:teuthology.orchestra.run.smithi118.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31305.log.gz 2024-08-18T13:53:15.700 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31606.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36045.log.gz 2024-08-18T13:53:15.700 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31606.log.gz 2024-08-18T13:53:15.700 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30167.log 2024-08-18T13:53:15.701 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37464.log 2024-08-18T13:53:15.713 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37932.log: /var/log/ceph/ceph-client.admin.30167.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37932.log.gz 2024-08-18T13:53:15.713 INFO:teuthology.orchestra.run.smithi118.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31498.log 2024-08-18T13:53:15.713 INFO:teuthology.orchestra.run.smithi118.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30167.log.gz 2024-08-18T13:53:15.713 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37192.log /var/log/ceph/ceph-client.admin.37464.log: 2024-08-18T13:53:15.713 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.31498.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32178.log 2024-08-18T13:53:15.713 INFO:teuthology.orchestra.run.smithi118.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37464.log.gz 2024-08-18T13:53:15.714 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33951.log 2024-08-18T13:53:15.714 INFO:teuthology.orchestra.run.smithi118.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31498.log.gz 2024-08-18T13:53:15.714 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37192.log: /var/log/ceph/ceph-client.admin.32178.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37192.log.gz 2024-08-18T13:53:15.714 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35559.log 2024-08-18T13:53:15.714 INFO:teuthology.orchestra.run.smithi118.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32178.log.gz 2024-08-18T13:53:15.714 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30740.log 2024-08-18T13:53:15.714 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.35559.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37246.log 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33951.log.gz 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35559.log.gz 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37273.log 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30740.log.gz 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37246.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37246.log.gz 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32205.log 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35726.log 2024-08-18T13:53:15.715 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37273.log: /var/log/ceph/ceph-client.admin.32205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37383.log 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37273.log.gz 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35726.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32205.log.gz 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31633.log 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35726.log.gz 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37383.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37383.log.gz 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31078.log 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38667.log 2024-08-18T13:53:15.716 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.31078.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31633.log.gz 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31078.log.gz 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35586.log 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.38667.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36466.log 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38667.log.gz 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.35586.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35586.log.gz 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38910.log 2024-08-18T13:53:15.717 INFO:teuthology.orchestra.run.smithi118.stderr:gzip /var/log/ceph/ceph-client.admin.36466.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32585.log 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36466.log.gz/var/log/ceph/ceph-client.admin.38910.log: 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38910.log.gz 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33980.log 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32585.log: /var/log/ceph/ceph-client.admin.37876.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32585.log.gz 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39263.log 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-08-18T13:53:15.718 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30304.log 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7%/var/log/ceph/ceph-client.admin.39263.log: -- replaced with /var/log/ceph/ceph-client.admin.33980.log.gz 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24418.log 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39263.log.gz 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37163.log 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30304.log: /var/log/ceph/ceph-client.admin.24418.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.24418.log.gz 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.30304.log.gz 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32558.log 2024-08-18T13:53:15.719 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37300.log 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.32558.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37163.log.gz 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38371.log 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32558.log.gz 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.37300.log: /var/log/ceph/ceph-client.admin.38371.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37300.log.gz 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29835.log 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38371.log.gz 2024-08-18T13:53:15.720 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33730.log 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.29835.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36770.log 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29835.log.gz 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.33730.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33730.log.gz 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.36770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39155.log 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36770.log.gz 2024-08-18T13:53:15.721 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30355.log 2024-08-18T13:53:15.722 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.39155.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39155.log.gz 2024-08-18T13:53:15.722 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/ceph-client.admin.30355.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30355.log.gz 2024-08-18T13:53:15.724 INFO:teuthology.orchestra.run.smithi118.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-08-18T13:53:15.862 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-18T13:53:15.865 INFO:teuthology.orchestra.run.smithi178.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-18T13:53:16.296 INFO:teuthology.orchestra.run.smithi118.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-18T13:53:16.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-08-18T13:53:16.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-08-18T13:53:22.183 INFO:teuthology.orchestra.run.smithi118.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-18T13:53:22.867 INFO:teuthology.orchestra.run.smithi178.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-18T13:53:24.309 INFO:teuthology.orchestra.run.smithi118.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-18T13:53:24.655 INFO:teuthology.orchestra.run.smithi178.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-08-18T13:53:26.575 INFO:teuthology.orchestra.run.smithi178.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-08-18T13:53:28.480 INFO:teuthology.orchestra.run.smithi118.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-18T13:53:29.079 INFO:teuthology.orchestra.run.smithi118.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-18T13:53:29.471 INFO:teuthology.orchestra.run.smithi178.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-18T13:53:35.829 INFO:teuthology.orchestra.run.smithi178.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-08-18T13:53:37.229 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-08-18T13:53:37.230 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-08-18T13:53:37.230 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m21.678s 2024-08-18T13:53:37.230 INFO:teuthology.orchestra.run.smithi178.stderr:user 1m21.216s 2024-08-18T13:53:37.230 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m2.485s 2024-08-18T13:53:40.898 INFO:teuthology.orchestra.run.smithi118.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.0.32128.log.gz 2024-08-18T13:53:40.900 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-08-18T13:53:40.900 INFO:teuthology.orchestra.run.smithi118.stderr:real 0m25.350s 2024-08-18T13:53:40.900 INFO:teuthology.orchestra.run.smithi118.stderr:user 1m5.621s 2024-08-18T13:53:40.901 INFO:teuthology.orchestra.run.smithi118.stderr:sys 0m2.225s 2024-08-18T13:53:40.901 INFO:tasks.ceph:Archiving logs... 2024-08-18T13:53:40.901 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/remote/smithi118/log 2024-08-18T13:53:40.903 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-18T13:53:50.701 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/remote/smithi178/log 2024-08-18T13:53:50.702 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-18T13:54:01.827 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-18T13:54:01.837 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-18T13:54:01.837 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-18T13:54:01.840 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-18T13:54:01.927 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-18T13:54:01.937 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-18T13:54:01.938 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-18T13:54:01.940 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-18T13:54:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: remote refid st t when poll reach delay offset jitter 2024-08-18T13:54:01.962 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================== 2024-08-18T13:54:01.962 INFO:teuthology.orchestra.run.smithi118.stdout:+hv01.front.sepi 67.205.162.81 3 u 33 64 377 0.095 -0.479 0.093 2024-08-18T13:54:01.962 INFO:teuthology.orchestra.run.smithi118.stdout:+hv02.front.sepi 63.231.80.2 3 u 28 64 377 0.069 0.585 0.107 2024-08-18T13:54:01.963 INFO:teuthology.orchestra.run.smithi118.stdout:*hv03.front.sepi 74.6.168.72 3 u 17 64 377 0.075 0.203 0.129 2024-08-18T13:54:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2024-08-18T13:54:01.964 INFO:teuthology.orchestra.run.smithi178.stdout: remote refid st t when poll reach delay offset jitter 2024-08-18T13:54:01.964 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================== 2024-08-18T13:54:01.964 INFO:teuthology.orchestra.run.smithi178.stdout:+hv01.front.sepi 67.205.162.81 3 u 24 64 377 0.079 1.459 0.236 2024-08-18T13:54:01.965 INFO:teuthology.orchestra.run.smithi178.stdout:+hv02.front.sepi 63.231.80.2 3 u 21 64 377 0.085 2.212 0.358 2024-08-18T13:54:01.965 INFO:teuthology.orchestra.run.smithi178.stdout:*hv03.front.sepi 74.6.168.72 3 u 24 64 377 0.090 2.218 0.224 2024-08-18T13:54:01.965 INFO:teuthology.orchestra.run.smithi178.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2024-08-18T13:54:01.965 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-18T13:54:01.976 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-18T13:54:01.976 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-18T13:54:02.063 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-18T13:54:02.075 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-18T13:54:02.086 INFO:teuthology.task.internal:Duration was 1419.253399 seconds 2024-08-18T13:54:02.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-18T13:54:02.101 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-18T13:54:02.101 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-18T13:54:02.104 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-18T13:54:02.156 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-18T13:54:02.156 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2024-08-18T13:54:02.156 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-18T13:54:02.171 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-08-18T13:54:02.171 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-18T13:54:02.185 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-18T13:54:02.186 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-18T13:54:02.188 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-18T13:54:02.271 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-18T13:54:02.271 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-18T13:54:02.273 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-18T13:54:02.359 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-18T13:54:02.373 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-18T13:54:02.374 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-18T13:54:02.389 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-18T13:54:02.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-18T13:54:02.414 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-18T13:54:02.416 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-18T13:54:02.429 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2024-08-18T13:54:02.460 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-08-18T13:54:02.494 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-18T13:54:02.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:54:02.500 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-18T13:54:02.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T13:54:02.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-18T13:54:02.517 INFO:teuthology.task.internal:Transferring archived files... 2024-08-18T13:54:02.518 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/remote/smithi118 2024-08-18T13:54:02.518 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-18T13:54:02.548 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-18_13:11:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7859093/remote/smithi178 2024-08-18T13:54:02.548 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-18T13:54:02.577 INFO:teuthology.task.internal:Removing archive directory... 2024-08-18T13:54:02.578 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-18T13:54:02.587 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-18T13:54:02.595 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-18T13:54:02.606 INFO:teuthology.task.internal:Not uploading archives. 2024-08-18T13:54:02.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-18T13:54:02.620 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-18T13:54:02.620 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-18T13:54:02.625 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-18T13:54:02.628 INFO:teuthology.orchestra.run.smithi118.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 18 13:54 /home/ubuntu/cephtest 2024-08-18T13:54:02.633 INFO:teuthology.orchestra.run.smithi178.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 18 13:54 /home/ubuntu/cephtest 2024-08-18T13:54:02.636 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-18T13:54:02.654 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-18T13:54:02.691 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 1419.2533993721008 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi118 with status 128: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a17c67c052c1476b20f99d847823755b26703128 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=011c2c82cd404891a9543c2fd0899f3c status: fail success: false 2024-08-18T13:54:02.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-18T13:54:02.800 INFO:teuthology.run:FAIL